How did the New York Times manage to spend $40 million on its pay wall?

by philg

Aside from wondering who will pay more than the cost of a Wall Street Journal subscription in order to subscribe to the New York Times, my biggest question right now is how the NY Times spent a reported $40-50 million writing the code (Bloomberg; other sources are consistent). Google was financed with $25 million. The New York Times already had a credit card processing system for selling home delivery. It already had a database management system for keeping track of Web site registrants. What did they spend the $40-50 million on? A monster database server to keep track of which readers downloaded how many articles? They should already have been tracking some of that for ad targeting. In any case, a rack of database servers shouldn’t cost $40 million.

What am I missing?

[I built a pay wall back in 1995 for the MIT Press, restricting access to some of their journals, e.g., Cell, to individual subscribers and people whose IP addresses indicated that they were at institutions with site-wide subscriptions. I can’t remember exactly what I charged the Press, but it was only a few days of work and I think the invoice worked out to approximately $40 million less than $40 million.]

54 thoughts on “How did the New York Times manage to spend $40 million on its pay wall?”

  3. Maybe IBM was involved

    1990s. Air traffic control system. 100 PAGES of doc per line of code. Surprisingly, a failed project.

  5. You know the answer to this don’t you ?

    Large organisations (and the largest of all) are more concerned with avoiding disaster than they are concerned with excellence.

    If something takes twice as long or costs ten times as much that’s fine if it reduces the risk of a career ending disaster by 0.0001%.

    Consultants, reports, proof of concepts, steering committees etc etc all designed to help avoid disaster or avoid accountability in the case of disaster.

    Obviously it’s no mystery, but pointing out the absurdity of it in the stark way you have is valuable.

  7. I am reminded of the Obama VA “Blue Button” project – to allow vets to download their Healthcare records. It was done for “free” and took about 8 weeks for the VA, and as much again for MEdicaid – now 55M americans can download their healthcare records. Here in Australia, NEHTA (NAtional Ehealth Transition Agency) is spending $250M over 2 years to do the same thing…

  8. Skunkworks, added irrelevant spending, special ineterest groups. Oh wait, you’re talking about a private corp. Most likely it was catering for meetings.

  10. They probably spent $100-200k on the programmers and hardware and the balance on the marketing team.

  12. Perhaps their most popular blogger, Paul Krugman, convinced them that the Keynesian Multiplier effect of spending lots of money building infrastructure without regard to cost-effectiveness would lead to greater profits for them in the future. Other than that, I’d go with Sam’s response. Large organizations take forever to get things done even when the objectives are clear and the ROI is immediate and obvious (c.f. GM and NUMMI, http://www.thisamericanlife.org/radio-archives/episode/403/nummi).

  13. Alex, I do find it odd that Stephen Levitt hightailed to his own paywall free safe-haven shortly beforehand (for the Freakonomics brand which he has made much money on by… giving stuff away). Your Krugman explanation seems like a solid hypothesis to me!

  14. I bypass the pay wall on my browser simply by going to the url address bar, removing the “&gwh=[long-alphanumeric-code]” at the end of the address and hitting enter

  15. In fact, in addition to all the things you mentioned, the Times already even had a pay wall! They’ve been charging for certain “premium” services, such as crossword puzzles, for a long time.

  16. Sounds like the same problem the federal gov’t has w/the tax code…spent too much time and money developing a system that all the big players can work around.

  17. One theory that is well supported by the data so far about how this paywall works is that The Times has made the paywall easy to bypass on purpose in order to create a situation where only the people that most greatly favor convenience over price will pay for the service and almost everyone else will still have a way to view the content for free. This will give them the mix of high traffic plus some additional income from the people most able and willing to pay that they are hoping for. In other words, they are following the YMCA’s policy that “no one is excluded because of inability to pay”.

    But why they would pay $40 million to implement an apparently very simple function, that is still confusing to me.

  18. I had a similar reaction to The Daily costing $30 million to develop. I have no idea what they spent that on either.

    Are IT consulting companies completely fleecing these big publications?

  19. I thought of an even stranger twist to this story… they weren’t embarrassed by how much it cost. Assuming it is true, why would they disclose that adding a filter or two to their Web site cost more than developing Google, Twitter, and Facebook combined?

  20. C’mon – say hello to our good friend, Systems Integration.

    I imagine it’s a very flexible system, designed to evolve as the business needs change. And I imagine it’s deeply integrated into existing legacy systems (subscriber / circulation, online identity, website, archives, search, apps, etc). I’m sure there were a LOT of meetings, over a couple of years, to map out possibly approaches – that is, business approaches – and then a series of mockups, an ever-changing datamodel, a prototype or three, a proof of concept, a test bed, monitoring and analytics, etc.

    I mean, sure: you, me, and the kid next door can fire up Rails and get some sort of hacked-up thing built in a week, but it’s not going to handle tens of millions of customers and maybe hundreds of millions of pageviews a year. They’ve said they don’t want it to be rock-solid now; it’s designed to be porous. It’s a multi-year investment.

    What do you think would be a reasonable cost, to do both the business modeling as well as the data/UI/tracking/integration build?

  21. Michael: What would be a reasonable cost to build a system that handles hundreds of millions of page views a year? I built such a system: photo.net (about 700 million page views/year when I was last responsible for it, in 2007). Did we spend $40 million doing it? I don’t think so, since we never had $40 million. Basically as of 2007 we had about $100,000 worth of hardware in one rack. We had one full-time programmer, a part-time contract programmer, a part-time database administration, and a part-time sysadmin.

    I was just playing around with the paywall from two fresh browsers (without previous NY Times cookies). It does seem as though whenever the paywall interrupts one can simply strip the crud from the end of the URL and then keep reading. What’s a reasonable cost to build a system that weak? It is hard to say, since it is hard to know who would want such a thing.

    Let’s back up and assume that we built an actual working paywall… let’s say that we’re going to set this up in a separate database so that we don’t mess up the existing production systems. We’ll need one dbadmin/sysadmin for the database server. We’ll need one programmer for three months to build the filters and another programmer perhaps to build the subscription sales site. We’ll need five testers in India to make sure that it works in all browsers and in all situations. We’ll need one white hat security expert to try to break in and then tell the programmers how to fix the holes. If we assume the five testers in India cost as much one programmer in New York, that’s a total of five full-time equivalents for three months (15 programmer-months). Let’s say that with the cost of Manhattan office space, payroll taxes, and health insurance a programmer costs $20,000 per month, so that’s $300,000. We might need some user interface and graphic design consulting and perhaps a project manager, but I still can’t get to more than $400,000 plus maybe $100,000 for hardware, so $0.5 million total.

  22. Not to mention how a paywall will kill SEO, page views, and ad revenue based on click throughs. Those who don’t know how to snip the tail end of a URL will refrain from posting links to NYTimes articles on Twitter, Facebook, their own blog, etc.

    LOL

    Now, what can I sell to the NYTimes for $40 million. Surely I have some digital doodad around here somewhere, that took me only a few hours to device, eh?

  24. At roughly $1 per monthly unique (they have about 45 million uniques a month), I’m sure the cost was rationalized across the amount of money they might make in the future. But I agree, it’s hard to understand how they spent the money.

    While the social media companies are so focused on customer acquisition over profitability, it’s fascinating to see a company who already has the customers be so willing to lose them.

    The biggest problem with this new pay model is dedicated online Times readers like myself are now searching for alternative primary news sources. I don’t need the stress of thinking about the 20 free articles a month and I certainly think the pricing is very aggressive.

    I’m sure I’ll visit the NYT in the future, but it will never be so cheap to acquire customers like me again. While the print newspaper was part of my daily life when I was younger, I transitioned to online only. Never imagined that I would transition again to no NYT in any form.

    Schumpeter would be impressed by NYT management.

  25. Maybe they hired the same developers as the Color iPhone app that launched last week with $41 million in funding.

  26. I’m not a programmer, but I imagine it’s far more complex and expensive to create a structure that is intelligently and explicitly permeable, than something that is impermeable or permeable in a blunt way. People point out that a few lines of code can get around it. The vast majority of users would rather pay a little than start pumping in java snippets.

    This wasn’t a programming project. It was a high-stakes business development project. Elephants don’t give birth to ants.

  28. $1 million or so in extra hardware (just in case), some testing and the rest on meetings to talk about it.

  29. System integration doesn’t explain it. Yes, the paper subscription database has to be linked to the online paywall database, but that just means that when an update happens that is of interest to the paywall (subscription start or end are all that I can think of off the top of my head, but maybe subscription suspended) a message gets sent to the paywall database. So a couple of triggers and a round of regression testing. Doesn’t add much to philg’s costing.

  30. There’s the whole cost of the actual paywall, and there’s the additional cost of one and a half years of carrying it while endless meetings were held and research done on how to find a dumbed down enough explanation that Sulzbergr could finally understand and sign off on. He’s the dustiest book on the shelf from what I’ve seen of him.

  31. 40 million for a paywall that can be bypassed by pressing the “esc” key before the web page fully loads.

  32. Philip Greenspun! I haven’t read your musings in years, but all of your nytimes.com commentary has been brilliant and funny. The point about disregarding the newspaper news has been echoed by my PhD adviser throughout my rather long PhD… 🙂 Now they’ve made it a bit easier for me, and all for just $40mil!

    Now, I just wish reddit.com would put up a paywall… for my productivity’s sake.

  33. The New York Times is taking it hard:

    – their real estate deals, central since the 80s, have gone sour
    – their mergers & acquisitions in the 90s led to big write downs
    – print circulation is down 15-20% since the early 2000s

    They are having serious problems, and like most American companies facing serious problems they whine and whine and whine. The paywall is just a $40M public tantrum. They’ll do just about anything but improve their product. Look at Detroit. What did they do about VW? They whined. What did they do about Honda and Toyota? They whined. Yes, their cars are a lot better than the 70s, but you can still do better for the money.

    I hate to think that we’ll be getting all our news outsourced from India. I would have rather seen the Times spend the money to beef up its reporting, but they gave that up before they hired Judith Miller. Let’s hope the folks in Mumbai, or maybe the bloggers, can do a better job.

  34. Maybe it really didn’t cost $40 million, or hasn’t yet. Also, you mentioned photo.net getting 700million views a year. The bloomberg article mentions 45 million unique visitors per month. If each visitor is looking at 3 pages per month that is already much higher than photo.net .. and that isn’t including images … etc ..

    That said, 40-50 million seems high. Maybe that includes datacenter fees for the next 15 years or something.

  35. Kaleberg: Ouch! That’s harsh. Mostly I don’t see what the nytimes does to add value to the news. They seldom put the news in perspective to a greater extent than other sources. So they are therefore reporting the same facts about the same stories as everyone else, e.g., “the nuclear reactor in Japan did X today”. I don’t see how their slightly different wording is worth a high subscription fee (on top of the readers having to view ads as well). The mid-March New Yorker magazine had an article on the effort to deal with the Deepwater Horizon oil spill and it was better than all the New York Times’s coverage combined.

    So I’m happy to pay for New Yorker (about $50/year mailed to my house; don’t think I would pay $300 or whatever the Times is actually asking) because they offer something different than other publishers. Also the print format has convenience value for me. But I wouldn’t pay for a newspaper because (a) the activity of reading the news is a waste of time, and (b) newspapers offer a commodity product.

    [The part of the NY Times that most frequently rises above the commodity level is the Sunday paper, e.g., the articles in the Magazine and Book Review. Perhaps not coincidentally it is that Sunday paper that people have been most willing to pay for.]

  36. I can tell you, when I have it in my mind that some site is going to prevent me from reading an article, going forward I do my best to remember never to click on any of their links again. WSJ being one of them.

  37. Having worked for them its easy to understand. They equate cost with value. They have squandered millions. Their newsroom may be world class. Their business side is filled with pretentious mediocre talent. All led by Arthur…a silver spoon media moron.

  38. Philg: re: what the times does to add value: it gets the news with reporters. The nuclear reactor situation is widely reported and is occuring in a place with wide news coverage. The middle east wars, for example, are not widely covered. It’s the Times’ reporters who many times provide the primary reporting that other news organizations repeat. In addition, redundent primary reporter coverage of any story provides you and me some assurance that the story provided is something close to the facts.

    Regarding the project cost: the NYTimes has forever been viewed as an attractive cash cow for any technology venture (what was that awful MS venture? the NYTimes Reader?). Even though they’re late to the game here, they’re still falling prey to digital hucksters.

  39. Anyone who has worked on large, scalable enterprise IT systems can tell you that $40 million for this effort is not justified, period. Either the NYT IT leadership has all fleeced the company and are preparing to head off to their new Bahamas homes, or they got taken by some big consulting firm.

    John Gruber’s comments as to the confusing overpriced approach on the customer side are spot on. The execution of the paywall is the move of an entrenched, unaware, deeply ossified bureaucracy who believe that because they’ve always been at the perceived peak of U. S. journalism that they always will remain there. Dramatically overpaying for this effort? Tells you all you need to know.

  40. This is a great discussion. I’ve been doing internet projects for 17 years and I’ve never seen something so seemingly trivial the NYTimes Paywall cost so much — and I’ve worked on dozens of corporate projects for Fortune 100 companies.

    I would like to have micropayments for nytimes content. That would be much more flexible and equitable to me. I like browsing the times unfettered — it would be nice to only have to pay (transparently) when I dive deep into an article.

    I can understand the desire for the times to ascribe perceived value to their paper, but $35 a month is insane.

    Here is a likely scenario: I get my daily news now from all the other free sources. I will dive into the nytimes to read the excellent analysis and exquisite editorial only after a particular story in the news piques my interest. This is the “I’ll pick up a used copy of the Times in the leftovers stand later in the day at the coffee shop.” Four-hundred twenty dollars $840.00 is way more than I’ll pay for two years of anyone’s content.

    Thinking about the rest of my life… $16,800.00 to read the New York Times for the next forty years. I….just….can’t……

  41. I’m surprised it cost this much to implement, where did the money go? datacenters? marketing? No wonder they’re so committed to the screwy pricing scheme that they’re getting so much flack about, now they have this huge expense to account for. They need to just simplify the subscription model, have one relatively low flat rate that lets you access all their content from any device, and people will probably pay. As a further incentive, they could include some paying-only content, like premiere video content or HD video, some bonus you don’t otherwise access. Of course, this would probably be easier to implement if they weren’t $40 million in the hole on the project… that’s a lot of subscriptions they need to just break even.

  44. Steve-
    I’m not sure the Times reporters add value, in many cases recently they decrease the value of the Times. Take the recent film article discussing how the Depression generation was tight fisted when it came to films (despite this era having the highest % of movie going, and the highest % of disposable income spent on movies). Or if you wish a more documented easily linked case:
    http://www.huffingtonpost.com/melissa-kirsch/new-york-times-obituary-c_b_102444.html
    Most telling is the reason for their sloppiness:
    “[T]his correction was delayed because editors and a reporter did not follow through on the complaint.” If they can’t be bothered to follow through on a complaint from a dead mans son, who cares what they have to say? Despite being a self-defined liberal I don’t trust what the times has to say on either the left or the right because I don’t believe they actually care about getting it right.

    I’m not unwilling to pay for news, I do so from the Financial Times, who follow and break news stories. The NYTimes waits for something they deem worthy and then feel self satisfied when the plant their bad brand on it

  46. font9a: You want micropayments? You’re already making micropayments to the nytimes and any other Web site that contains ads… by downloading the ads at the same time that you read the articles. Want to pay them a little more? Click on a few ads.

    As ad targeting becomes more sophisticated with cross-site tracking, e.g., showing you ads for hotels and restaurants in Hawaii on all of the sites that you visit right after you’ve booked a plane ticket, I would think that the value of Web ads would go up. Thus an organization that wants more revenue from its readership need only wait a few years (or try to grow the size of the readership or number of page views).

  47. Dru

    “Anyone who has worked on large, scalable enterprise IT systems can tell you that $40 million for this effort is not justified, period. Either the NYT IT leadership has all fleeced the company and are preparing to head off to their new Bahamas homes, or they got taken by some big consulting firm. ”

    The Yorkshire Ranter has some comments expressing a contrary opinion: http://yorksranter.wordpress.com/2011/04/01/scaling-and-scoping-the-nyt-paywall/

    Daniel Davies does, as well:
    http://d-squareddigest.blogspot.com/2011/03/only-minority-of-talented-computer.html

  48. One comment by the Ranter:

    ” (Also, Greenspun’s project had to handle two different billing rules – are you a subscriber? are you an affiliate? Nothing like “are you a non-logged in member with 15 or more free articles remaining who is following a Twitter link to one of our news analysis pieces?”)”

  49. Barry: Thanks for the links. Some of what the Yorkshire Ranter says makes sense to me, particularly his focus on latency. On the other hand, the paywall is porous by design so the database need not be handled by an ACID-compliant system such as a standard disk-based RDBMS. Nor does it need to become a point of failure. If the paywall is broken, the core site can continue functioning by delivering pages to all and sundry. We built some of that robustness into photo.net by serving pages from the file system and comments from the RDBMS. If the RDBMS connections (from a common pool) were all in use we gracefully degraded by offering links to a page that would query the comments out rather than querying the comments out and showing the in-line.

    As long as one is willing to accept a risk of serving a page to someone who didn’t pay (and they’ve already accepted that risk by letting in folks from Google, Twitter, etc.), the paywall does not have to be engineered to telco standards. A mobile phone carrier is not willing to accept the risk that you make a 2-hour call to a mobile phone in France and they incur charges of $200 while the record gets lost and they have no way to bill you. That’s a very different situation than the Times delivering Paul Krugman’s or David Brooks’s wisdom to someone who is unworthy of it.

  51. Justin: If I may take a stab at that… suppose we assume an average monthly subscription price of $20 (iPhone is $15, iPad only is $20, and the combo deal is $35 – by the way, kudos to NYT from the great price break deal on the iPhone/iPad combo). $40M paywall cost / $20 monthly subscription = 2M subscriber-months required to pay for the cost of the paywall development.

    Looking at circulation numbers, NYT has a print circulation of ~1.4M on weekends and 900k on weekdays (source: http://www.nytimes.com/2010/04/27/business/media/27audit.html)

    Pulling some numbers from the web, Wired (800k print circulation) sold 24k iPad magazine apps in 24 hours. This was a $5 app with no subscription and was purchased by around 3% of print subscribers, including a disproportionately large number of early-adopter geeks as Wired was the first magazine available on the iPad. It took six weeks for Popular Science (1.3M print circulation) to sell 10k subscriptions, meaning that 0.7% of subscribers signed up on the iPad. (source: http://en.wikipedia.org/wiki/List_of_magazines_by_circulation)

    Judging very roughly from the Wired and Popular Science numbers, NYT would be very optimistic to expect that 5-10% of its subscribers might pay for the online edition. Assuming that 10% of 1M subscribers cough up their $20/month, it will take just under two years for NYT to recoup the $40M.

  52. In all of the speculation about the NYT payment system, two facts stand out:

    * They have not announced how much they spent. The $40m figure cited by Bloomberg is unsourced.

    * Very few people understand the complexity of what’s being deployed: “an apparently very simple function, that is still confusing to me.” It is not simple.

    The NYT system is a business experiment. A business experiment isn’t just an academic exercise — it’s a financial gamble with the hope of a payback. To work, it needs metrics to let the operators know what’s going on, and levers to adjust the system to what’s been learned. You don’t just launch and record the results.

    The objective is to optimize revenue from committed readers by behaviorally targeting them with a request for payment, without turning away occasional visitors and without materially damaging traffic. The model allows free pageviews up to a point, followed by a payment challenge. That sounds simple but it’s not.

    From a business perspective, it’s a very, very tricky balancing act to pull off, and there are huge risks. You can’t figure out the right settings — what’s the optimum number of free visits per month? — on a whiteboard or in a committee meeting. If you’re gambling the future of an enterprise like the New York Times, you’re going to go in armed with tons of market research. There are dozens of relevant variables. What content should be measured, an what should be totally free? What about multiple device usage, mobile devices, tablets, apps?

    All of this requires a customer service organization to support users who will be paying and undoubtedly having problems, since computers are still unreliable time-sucking helldemons. The technology has to perform in microseconds and scale to meet not only the huge gross traffic of the Times’ website, but also the spikes that will occur at the next major national emergency. And it requires a communication plan. (Advertising is expensive. That’s why the newspaper sells it.)

    Those who have pointed to organizational risk-aversion and the challenges of systems integration are undoubtedly correct. All projects in large organizations suffer from scope creep. Tying online usage to print subscribership to mobile app purchases to tablet subscriptions is a recipe for headaches.

    Without question, two guys in a garage could launch a paid-content website for peanuts. This is not that website.

  53. Adam: “Judging very roughly from the Wired and Popular Science numbers, NYT would be very optimistic to expect that 5-10% of its subscribers might pay for the online edition. Assuming that 10% of 1M subscribers cough up their $20/month, it will take just under two years for NYT to recoup the $40M.”

    Good catch – in addition (as yelvington sorta pointed out), a big concern is how this affected NYT’s ability to sell ads, and how much they could charge for those ads. The subscribers might be a premium market, and could be served the appropriate ads. Market research on the expected changes in readership and ad revenue might have been the biggest cost here.

    PhilG: “A mobile phone carrier is not willing to accept the risk that you make a 2-hour call to a mobile phone in France and they incur charges of $200 while the record gets lost and they have no way to bill you. ”

    Actually, I’m sure that they do. If they have very, very high rates of up-time, that tells you on which side of the equation they leaned towards. Presumably systems which set an extremely low risk of not billing would have more problems.

Comments are closed.