{"id":39,"date":"2005-03-07T21:24:31","date_gmt":"2005-03-08T01:24:31","guid":{"rendered":"http:\/\/philip.greenspun.com\/blogtest\/2005\/03\/07\/how-long-is-the-average-internet-di"},"modified":"2005-03-07T21:24:31","modified_gmt":"2005-03-08T01:24:31","slug":"how-long-is-the-average-internet-discussion-forum-posting","status":"publish","type":"post","link":"https:\/\/philip.greenspun.com\/blog\/2005\/03\/07\/how-long-is-the-average-internet-discussion-forum-posting\/","title":{"rendered":"How long is the average Internet discussion forum posting?"},"content":{"rendered":"

<\/a><\/p>\n

A friend of mine who works at a database management system company asked for thoughts on how long a string a database table needs to be able to store, as a practical matter, to serve most Internet programming needs.  This prompted me to do some queries into the photo.net discussion forum.  Here’s my message to him, which I thought would be interesting to nerd readers….<\/P>
\n

Three basic issues for Web development relating to varchar\/clob:<\/P>
\n

1) Strings are uploaded from a browser-rendered TEXTAREA are of a length that is impossible for the programmer to predict. In a sense, then, every text slot in the database must be prepared to accept a string of arbitrary length. <\/FONT>http:\/\/www.cs.tut.fi\/~jkorpela\/forms\/textarea.html#browserlimits<\/U><\/FONT><\/A> reveals<\/A> that some browsers have limits of 32K or 64K but that as Microsoft and Mozilla get more sophisticated these seem to be disappearing.<\/P>
\n

2) Software developers of Internet applications are often first-time SQL programmers and sometimes first-time programmers altogether. Unless a DBMS can make CLOBs work with every SQL function and command these novice programmers must learn a whole new computer language, essentially, to deal with CLOBs.<\/P>
\n

3) Internet applications are often developed using feeble ad hoc tools, such as PHP (my students in 6.171, MIT’s Software Engineering for Internet Applications, mostly picked PHP to do their semester project even though I would have discouraged this, being a mistruster of thrown-together unnecessary new languages). Many of these tools don’t have facilities for dealing with anything beyond the basic SQL data types so they couldn’t use CLOBs if they wanted to.<\/P>
\n

I think for Web development it is reasonable to expect the average string length from the user to be 300 chars, despite wanting to be prepared for a maximum of 32K or even larger. Oops. Typing that prompted me to do the query (see below). Averaging 2 million messages on photo.net, the correct number is 425. The histogram query reveals that out of 2 million messages over a 10-year period a 32K limit would have resulted in 6 messages being rejected and a 16K limit something like 30 rejections.<\/P>
\n

If you wanted to implement something like Salon.com as a single RDBMS table for both articles and comments on articles I think a 64K limit might be required. If someone authors a 5000-word magazine article in Microsoft Word and then saves as HTML that will be 25-30k of content plus at least a factor of 2 in HTML tags and other Microsoft-added filler. <\/FONT>http:\/\/www.photo.net\/bboard\/q-and-a-fetch-msg?msg_id=002oFh<\/U><\/FONT><\/A> is an example of one of the big postings on photo.net. It is 37962 chars long, the HTML is very clean (i.e., much less filler than if saved by Word), and yet the page doesn’t seem excessively long.<\/P>
\n

So… my conclusion from looking at the queries below is that 32K would do the job for a pure discussion forum system and that it would be marginal for storing articles unless a publisher decided that everything should be broken up into “part 1”, “part 2”, and “part 3”. Looking at the .html files on photo.net, the vast majority are in fact under 32K and most are under 64K.<\/P>
\n

However, if you look at <\/FONT>http:\/\/philip.greenspun.com\/seia\/<\/U><\/FONT><\/A> the largest chapters are 88.7k.<\/P>
\n

If you want to facilitate novice programmers building full-scale content management systems where all the content is uploaded from browsers it might be necessary to make the varchar datatype bloat up to 100k-ish. But 32K would be adequate to build something like eBay, amazon (user-uploaded content and much of the publisher content as well), or photo.net discussion forums.<\/P>
\n

———– some stats from photo.net<\/P>
\n

select avg(dbms_lob.getlength(MESSAGE)),count(*) from bboard;<\/FONT><\/P>
\n

AVG(DBMS_LOB.GETLENGTH(MESSAGE))   COUNT(*)
——————————– ———-
        424.672669    2052290<\/FONT><\/P>
\n

select round(dbms_lob.getlength(MESSAGE),-3), count(*)
from bboard
group by round(dbms_lob.getlength(MESSAGE),-3);<\/FONT><\/P>
\n


ROUND(DBMS_LOB.GETLENGTH(MESSAGE),-3) COUNT(*)
————————————- ———-
        0  1452035
     1000   510236
     2000    58330
     3000    11972
     4000     3399
     5000     1303
     6000      481
     7000      264
     8000      138
     9000       93
    10000       66<\/FONT>
    11000       38
    12000       22
    13000       21
    14000       17
    15000        6
    16000       12
    17000        9
    18000       14
    19000        4
    20000        4
    21000        1<\/FONT>
    22000        2
    23000        1
    24000        4
    25000        3
    26000        2
    27000        1
    30000        2
    32000        1
    33000        2
    34000        1
    37000        1<\/FONT>
    38000        1<\/FONT><\/P>
\n

Note:  Oracle did a much better job formatting these in SQL*Plus; for some reason the tabs didn’t carry through after cutting and pasting.<\/FONT><\/P>
\n

——————————<\/FONT><\/P><\/FONT><\/FONT>
\n

Epilogue (not from my email to the friend):<\/P>
\n

Look how much fun it is to program SQL.  Three lines of code and you get an interesting answer (and those three lines would have been much cleaner and simpler if we hadn’t been forced to use the CLOB datatype, which has its own strange accessor functions).  Compare to Java and C where typing until your fingers fall off usually doesn’t result in much of anything.  SQL, Lisp, and Haskell are the only programming languages that I’ve seen where one spends more time thinking than typing.<\/P><\/p>\n","protected":false},"excerpt":{"rendered":"

A friend of mine who works at a database management system company asked for thoughts on how long a string a database table needs to be able to […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_jetpack_newsletter_access":"","_jetpack_dont_email_post_to_subs":false,"_jetpack_newsletter_tier_id":0,"_jetpack_memberships_contains_paywalled_content":false,"_jetpack_feature_clip_id":0,"_jetpack_memberships_contains_paid_content":false,"footnotes":"","jetpack_post_was_ever_published":false},"categories":[1],"tags":[],"class_list":["post-39","post","type-post","status-publish","format-standard","hentry","category-uncategorized"],"yoast_head":"\nHow long is the average Internet discussion forum posting? - Philip Greenspun\u2019s Weblog<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/philip.greenspun.com\/blog\/2005\/03\/07\/how-long-is-the-average-internet-discussion-forum-posting\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"How long is the average Internet discussion forum posting? - Philip Greenspun\u2019s Weblog\" \/>\n<meta property=\"og:description\" content=\"A friend of mine who works at a database management system company asked for thoughts on how long a string a database table needs to be able to […]\" \/>\n<meta property=\"og:url\" content=\"https:\/\/philip.greenspun.com\/blog\/2005\/03\/07\/how-long-is-the-average-internet-discussion-forum-posting\/\" \/>\n<meta property=\"og:site_name\" content=\"Philip Greenspun\u2019s Weblog\" \/>\n<meta property=\"article:published_time\" content=\"2005-03-08T01:24:31+00:00\" \/>\n<meta name=\"author\" content=\"philg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"philg\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"6 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/2005\\\/03\\\/07\\\/how-long-is-the-average-internet-discussion-forum-posting\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/2005\\\/03\\\/07\\\/how-long-is-the-average-internet-discussion-forum-posting\\\/\"},\"author\":{\"name\":\"philg\",\"@id\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/#\\\/schema\\\/person\\\/448e4e6a0ae9105741a06dcf62cee50e\"},\"headline\":\"How long is the average Internet discussion forum posting?\",\"datePublished\":\"2005-03-08T01:24:31+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/2005\\\/03\\\/07\\\/how-long-is-the-average-internet-discussion-forum-posting\\\/\"},\"wordCount\":1135,\"commentCount\":25,\"publisher\":{\"@id\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/#\\\/schema\\\/person\\\/448e4e6a0ae9105741a06dcf62cee50e\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"CommentAction\",\"name\":\"Comment\",\"target\":[\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/2005\\\/03\\\/07\\\/how-long-is-the-average-internet-discussion-forum-posting\\\/#respond\"]}]},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/2005\\\/03\\\/07\\\/how-long-is-the-average-internet-discussion-forum-posting\\\/\",\"url\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/2005\\\/03\\\/07\\\/how-long-is-the-average-internet-discussion-forum-posting\\\/\",\"name\":\"How long is the average Internet discussion forum posting? - Philip Greenspun\u2019s Weblog\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/#website\"},\"datePublished\":\"2005-03-08T01:24:31+00:00\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/2005\\\/03\\\/07\\\/how-long-is-the-average-internet-discussion-forum-posting\\\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/2005\\\/03\\\/07\\\/how-long-is-the-average-internet-discussion-forum-posting\\\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/2005\\\/03\\\/07\\\/how-long-is-the-average-internet-discussion-forum-posting\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"How long is the average Internet discussion forum posting?\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/\",\"name\":\"Philip Greenspun\u2019s Weblog\",\"description\":\"A posting every day; an interesting idea every three months...\",\"publisher\":{\"@id\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/#\\\/schema\\\/person\\\/448e4e6a0ae9105741a06dcf62cee50e\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":[\"Person\",\"Organization\"],\"@id\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/#\\\/schema\\\/person\\\/448e4e6a0ae9105741a06dcf62cee50e\",\"name\":\"philg\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/wp-content\\\/uploads\\\/2025\\\/11\\\/20210525-philip-greenspun-sea-turtle-release-scaled.jpg\",\"url\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/wp-content\\\/uploads\\\/2025\\\/11\\\/20210525-philip-greenspun-sea-turtle-release-scaled.jpg\",\"contentUrl\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/wp-content\\\/uploads\\\/2025\\\/11\\\/20210525-philip-greenspun-sea-turtle-release-scaled.jpg\",\"width\":2047,\"height\":2560,\"caption\":\"philg\"},\"logo\":{\"@id\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/wp-content\\\/uploads\\\/2025\\\/11\\\/20210525-philip-greenspun-sea-turtle-release-scaled.jpg\"},\"url\":\"https:\\\/\\\/philip.greenspun.com\\\/blog\\\/author\\\/philg\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"How long is the average Internet discussion forum posting? - Philip Greenspun\u2019s Weblog","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/philip.greenspun.com\/blog\/2005\/03\/07\/how-long-is-the-average-internet-discussion-forum-posting\/","og_locale":"en_US","og_type":"article","og_title":"How long is the average Internet discussion forum posting? - Philip Greenspun\u2019s Weblog","og_description":"A friend of mine who works at a database management system company asked for thoughts on how long a string a database table needs to be able to […]","og_url":"https:\/\/philip.greenspun.com\/blog\/2005\/03\/07\/how-long-is-the-average-internet-discussion-forum-posting\/","og_site_name":"Philip Greenspun\u2019s Weblog","article_published_time":"2005-03-08T01:24:31+00:00","author":"philg","twitter_card":"summary_large_image","twitter_misc":{"Written by":"philg","Est. reading time":"6 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/philip.greenspun.com\/blog\/2005\/03\/07\/how-long-is-the-average-internet-discussion-forum-posting\/#article","isPartOf":{"@id":"https:\/\/philip.greenspun.com\/blog\/2005\/03\/07\/how-long-is-the-average-internet-discussion-forum-posting\/"},"author":{"name":"philg","@id":"https:\/\/philip.greenspun.com\/blog\/#\/schema\/person\/448e4e6a0ae9105741a06dcf62cee50e"},"headline":"How long is the average Internet discussion forum posting?","datePublished":"2005-03-08T01:24:31+00:00","mainEntityOfPage":{"@id":"https:\/\/philip.greenspun.com\/blog\/2005\/03\/07\/how-long-is-the-average-internet-discussion-forum-posting\/"},"wordCount":1135,"commentCount":25,"publisher":{"@id":"https:\/\/philip.greenspun.com\/blog\/#\/schema\/person\/448e4e6a0ae9105741a06dcf62cee50e"},"inLanguage":"en-US","potentialAction":[{"@type":"CommentAction","name":"Comment","target":["https:\/\/philip.greenspun.com\/blog\/2005\/03\/07\/how-long-is-the-average-internet-discussion-forum-posting\/#respond"]}]},{"@type":"WebPage","@id":"https:\/\/philip.greenspun.com\/blog\/2005\/03\/07\/how-long-is-the-average-internet-discussion-forum-posting\/","url":"https:\/\/philip.greenspun.com\/blog\/2005\/03\/07\/how-long-is-the-average-internet-discussion-forum-posting\/","name":"How long is the average Internet discussion forum posting? - Philip Greenspun\u2019s Weblog","isPartOf":{"@id":"https:\/\/philip.greenspun.com\/blog\/#website"},"datePublished":"2005-03-08T01:24:31+00:00","breadcrumb":{"@id":"https:\/\/philip.greenspun.com\/blog\/2005\/03\/07\/how-long-is-the-average-internet-discussion-forum-posting\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/philip.greenspun.com\/blog\/2005\/03\/07\/how-long-is-the-average-internet-discussion-forum-posting\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/philip.greenspun.com\/blog\/2005\/03\/07\/how-long-is-the-average-internet-discussion-forum-posting\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/philip.greenspun.com\/blog\/"},{"@type":"ListItem","position":2,"name":"How long is the average Internet discussion forum posting?"}]},{"@type":"WebSite","@id":"https:\/\/philip.greenspun.com\/blog\/#website","url":"https:\/\/philip.greenspun.com\/blog\/","name":"Philip Greenspun\u2019s Weblog","description":"A posting every day; an interesting idea every three months...","publisher":{"@id":"https:\/\/philip.greenspun.com\/blog\/#\/schema\/person\/448e4e6a0ae9105741a06dcf62cee50e"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/philip.greenspun.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":["Person","Organization"],"@id":"https:\/\/philip.greenspun.com\/blog\/#\/schema\/person\/448e4e6a0ae9105741a06dcf62cee50e","name":"philg","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/philip.greenspun.com\/blog\/wp-content\/uploads\/2025\/11\/20210525-philip-greenspun-sea-turtle-release-scaled.jpg","url":"https:\/\/philip.greenspun.com\/blog\/wp-content\/uploads\/2025\/11\/20210525-philip-greenspun-sea-turtle-release-scaled.jpg","contentUrl":"https:\/\/philip.greenspun.com\/blog\/wp-content\/uploads\/2025\/11\/20210525-philip-greenspun-sea-turtle-release-scaled.jpg","width":2047,"height":2560,"caption":"philg"},"logo":{"@id":"https:\/\/philip.greenspun.com\/blog\/wp-content\/uploads\/2025\/11\/20210525-philip-greenspun-sea-turtle-release-scaled.jpg"},"url":"https:\/\/philip.greenspun.com\/blog\/author\/philg\/"}]}},"jetpack_sharing_enabled":true,"jetpack_shortlink":"https:\/\/wp.me\/paokyZ-D","jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/philip.greenspun.com\/blog\/wp-json\/wp\/v2\/posts\/39","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/philip.greenspun.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/philip.greenspun.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/philip.greenspun.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/philip.greenspun.com\/blog\/wp-json\/wp\/v2\/comments?post=39"}],"version-history":[{"count":0,"href":"https:\/\/philip.greenspun.com\/blog\/wp-json\/wp\/v2\/posts\/39\/revisions"}],"wp:attachment":[{"href":"https:\/\/philip.greenspun.com\/blog\/wp-json\/wp\/v2\/media?parent=39"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/philip.greenspun.com\/blog\/wp-json\/wp\/v2\/categories?post=39"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/philip.greenspun.com\/blog\/wp-json\/wp\/v2\/tags?post=39"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}