Microsoft's "I mean it" content-type parameter

It would be very fun to see the example they cite, I sincerely doubt they
exist to any legitimate extent today. friends crawling the web could
probably give us hard numbers. I suspect the short history goes;
>
* some folks start serving files over http:, associate .html with text/html
>
* 1000's download ms authoring tools to create default.htm files
>
* 100's uploading to these "ancient" servers discover they render as either
binary/octet-stream or text/plain
>
* MS fixes their client to display .htm files as html
>
Interestingly, they don't work around the fact that all of these servers are
also configured to serve index.html and not default.htm. If they relied on
the administrators to fix one side of the coin

William -

There are tons of legitimate use cases here they you have completely overlooked. For example, lots of server side applications throw out content of a type different from what their file extension would indicate. For example, the earliest "hit counter" programs were .cgi or pl files (typically) generating image/gif or image/jpeg content. The Web servers were set up explicitly to serve the output of those applications as text/html. And a great many developers had no idea that they needed to change the Content-type at the code level to make this work. Content sniffing made life easier for these developers. Indeed, Content-disposition is a brutally critical header for any developer making, say, a file download application that actually spews forth the bits itself instead of performing a redirection.

J.Ja

Thu, 3 Jul 2008, Justin James wrote:

There are tons of legitimate use cases here they you have completely
overlooked. For example, lots of server side applications throw out content
of a type different from what their file extension would indicate. For
example, the earliest "hit counter" programs were .cgi or .pl files
(typically) generating image/gif or image/jpeg content. The Web servers were
set up explicitly to serve the output of those applications as text/html.
And a great many developers had no idea that they needed to change the
Content-type at the code level to make this work. Content sniffing made life
easier for these developers.

Uh, that doesn't make sense.

Sure, some scripts output wrong Content-Type. Then no browser can output it
correctly and thus you fix the server side.

But, this system with bad Content-Type outputs still showing up nicely only
works if the client *already* have does this "sniffing" business and thus they
more or less encouraged the server-side hackers to remain sloppy.

So this cannot have been a case where the browser adapted to how servers work,
since servers would hardly ever have worked this way if some browsers didn't
already support it

I find this "I promise this time I really mean that the type is what I say"
attribute hilariously funny.

--

/ daniel.haxx.se

Justin James wrote:
The *entire* Web is founded on sloppy programmers,

Indeed, just look at HTML5 potentially taking over from XHTML. It's
practically reincarnating sloppiness, admitting defeat with XHTML,
this time trying to formalise how to handle slop so at least we do it
all the same.

(But last time I looked, it didn't handle slop the same way as certain
major browsers, but rather according to what the HTML5 authors thought
would be sensible, so that part seems doomed to be not implemented
according to spec, but as Yet Another incompatible compatibility layer
in real browsers.)

-- Jamie

>There are
>situations where it doesn't. The only way to resolve it is to have a flag
>that triggers a "no sniffing mode"; to do it the other way around (with a
>flag that *turns on* sniffing mode) would contradict existing behavior and
>therefore Break The Web.

Nonsense. HTTP/1.1 defined the mechanism to do just this. The fact that
vendors ignored this, suffered the consequences on vuln-dev and bugtraq,
and will continue to do so until they follow the protocol reiterates that
sniffing has a place, and within a well defined protocol this isn't it.

It is quite clear that you are ignoring the point here. The point is *not* what the spec says. As you point out, there is a serious disconnect between reality and the spec. What you are essentially saying is, "if everyone just followed the spec, everything would be fine." Which is true. But it is also not what happened. Which is the point.

Getting the currently HTML spec to handle the current reality in a way that not only leaves existing applications un-broken (regardless of their usage of a non-standard behavior) while also providing an option for avoiding the non-standard behavior is the challenge here. I am curious what you think would be a good approach to resolving this situation as it currently stands. And just re-writing the browsers to stop content sniffing is not a realistic answer, nor one that meets the goals set forth.

J.Ja

Sniffing content causes issues, for example when you want to serve an
HTML file with text/plain on *purpose*. Use case: insert the source
code of an html document with object or iframe sent as text/plain.

Yes, this is correct. But it does not contradict my statement that this
proposal does not break anything. If anything, it lends weight to the
proposal. After all, browsers are performing sniffing anyways already,
*regardless of whether or not they are supposed to* (a phrase that can be
applied to much of browsers' behavior). Therefore, this proposal provides
a mechanism for people on the server side to override that behavior in
precisely the scenarios that you describe.

There are situations where content sniffing makes sense. There are
situations where it doesn't. The only way to resolve it is to have a flag
that triggers a "no sniffing mode"; to do it the other way around (with a
flag that *turns on* sniffing mode) would contradict existing behavior and
therefore Break The Web.

J.Ja

Rather than persisting FUD, I'd challenge you to point out only one
significant site, and a relatively minor site, affected by such a change.

on the list have already provided some pretty decent examples, no need for me to do so. While those sites may constitute a minority of the total number of Web sites out there, they are some fairly major sites. Furthermore, as we have seen in past discussions on this mailing list, if a change we make "breaks" 0.1% of pages, that is still millions of pages, more than we usually like to "break".

Folks who insist that sniffing is "necessary" really aught to back up the
assertion with hard data, or close the significant vulnerabilities that
persist in the ecosystem.

As a technique, it is not "necessary". However, not breaking the Web *is* necessary, or else HTML 5 will become yet another ignored spec on the pile. So it is now important that we find a way to deal with this without causing *more* problems.

As mentioned in a previous note, sniffing served a noble purpose for a safer
environment, one that simply doesn't exist.

I never disagreed with you here. At best, I think people on this list can provide a guess as to the historical thinking that lead to browsers doing this. You can go ahead and stick to the position of "I don't care about anyone not following the spec, and I wish browsers wouldn't enable them to write bad code." In which case you back a vision of the HTML spec that stands in an ivory tower, or is some sort of brass ring (wow, I just realized the M:tG references there!) attainable by only the elite who use The Right Software and who have memorized many hundred-plus page specs. , you can back a more pragmatic vision, where we work hard to get a spec that Real World Developers can target. Personally, as a "real world developer", I opt for the latter.

Also, I would greatly appreciate it if you didn't toss FUD accusations around. First of all, personal attacks are highly unnecessary. Second of all, I am one of the people active on this list without any strong ties to an industry player. I do not represent a major company. I am not trying to get my employers decisions ratified after the fact by the HTML 5 spec. In fact, my current employer has no intention of using either HTTP or HTML in any of their products, except for possibly as a documentation format (doubtful at that). I chose to join this group since I felt that the HTML 5 draft that I read was headed in the wrong direction. The "special interest" that I try my best to represent are real world developers who have to use this spec and HTML documents, based upon my own experiences with HTML, HTTP, etc. and et al. Given my reasons for participation, I have zero reason to "persist FUD" (as you state). Finally, I am not "persisting FUD" anyways.

So, now that your challenge to me has been answered by others, I once again ask you to present your own solution to the problem. Not, "well, if only the evil browsers from vendors I don't like would only follow the spec to the letter, everything would be fine!" But an actual alternative to what has already been presented (the humorously, but accurately titled "Microsoft's 'I mean it' content-type parameter").

J.Ja

Eric Lawrence wrote:

keep in mind, however, that most folks (even the ultra-web engaged
on these lists) see but a small fraction of the web, especially
considering private address space/intranets, etc.

Yes, but a general philosophical problem with any "do what I mean"
flag applies:

(1) A relevant fraction of the Web got it wrong, using x=y where
they should have said x=z.
(2) Therefore you couldn't trust that x=y means x=y, introducing
some "what is x divination".
(3) That annoyed another relevant fraction of the Web who really
want x=y when they say x=y.
(4) You add an "I mean it" flag for (3), sticking to "divination"
as default for (1).
(5) In theory protocols, software, and config files are upgraded
to add those new "I mean it" flags everywhere. As that is a
worldwide upgrade stunt you lose a major fraction of the Web
sticking to (1) or (3) without this flag.
(6) Another major fraction does what you want, among them a part
of (1) now saying "x=y I mean it" when they clearly want x=z.
(7) SNAFU, your flag made it worse.

Some problems can't be solved in specifications because it's a
problem with folks never reading specifications.

Frank