As Google hires all of the world’s good software engineers and my friends with startup companies fight over the scraps I am left to wonder how everyone could have been so wrong in predicting that the world would be glutted with good programmers and sysadmins by now.
In the early days of programming there was not considered to be a limit on how many people could be trained to do the job. IBM would hire graduates from all kinds of college majors, e.g., music, and train them (there were hardly any computer science programs at universities back in the 1950s). A popular idea among corporate managers is that this could be a good job for women, who were readily available at reasonable rates.
With the rise of the worldwide Internet, open-source, and inexpensive microprocessors, it seemed inevitable that the world would be glutted with technically skilled people. Programming is easier to learn than a foreign language (a friend who has a PhD in physics and is an expert hardware and software developer says that learning Mandarin and to read Chinese was by far the hardest thing that he ever did). Getting hired as a programmer does not require a specific degree or training course. Programming or system administration can be learned by anyone worldwide who has access to a $300 personal computer and an Internet connection. Why wouldn’t the millions of unemployed Americans train themselves to code? Why wouldn’t middle class people in China, India, and Africa?
Admittedly in the U.S. folks with an organized mind, the grit to get through technical subjects, and the drive to go to work every day can find higher paying jobs that involve more social interaction (e.g., medical doctor, Wall Street banker, etc.). But that doesn’t explain why people in the Philippines or China aren’t training themselves en masse to be able to soak up the $30-100/hour jobs that would be readily available to them if they could demonstrate the ability to turn a customer request into working code. Why are the Greeks, Portuguese, and Spaniards going bankrupt instead of learning to administer Cisco and Linux?
Any good futurist circa 1990 would have predicted that by now most of the world’s programming would be happening in low-to-medium-wage countries and that customers wouldn’t have to pay a lot for high quality software. But it hasn’t happened (e.g., see my posting about California’s state government spending $327 million on a straightforward ecommerce site (I visited http://www.coveredca.com just now and it seems to be a work in progress)). Why instead do we have catfights over mediocre talent in Kendall Square and Silicon Valley?
You just need to visit The Daily WTF for your answer.
The floor for writing decent software is probably an IQ of 125. Over 125 is a much smaller share of the global population than you likely think it is. To be a programmer you also have to be a weird person willing to sit frustrated with horribly broken tools and materials, alone in front of a screen for very long stretches. Most people can’t swallow that.
Or, equally, Reddit/HN, and join the intellectual hunger strike.
Given that there isn’t a glut (and in fact there’s a shortage) why haven’t programming salaries reflected the law of supply and demand? Given enough of an incentive, the supply aught to catch up?! Instead, there’s a push to keep salaries down and increase H1B’s (or whatever they’re called these days)
There’s a possible reason.
What is the metric of “good” and what is the evidence about the size of that population, big or small? Are we reasoning from anecdotes here? (I.e. who google gets vs. what startups can find)
Task organization in software development makes a big difference. I’ve seen teams where you have one smart architect who spec’s everything out so tightly, the programmers are essentially smart typewriters who know java syntax. They’re not really doing much in the way of design, algorithms, or problem solving. They’re implementing a function that takes a void argument, and returns a String with the value “Hello, World”.
Skill seems to me to be so variable, that companies would do well to try and reduce the amount of skill the average software developer would have to have in order to contribute meaningfully. Good programmers can, after all, be too good — reminds me of what I heard about perl recently. “Since debugging code is always at least twice as hard as writing the code in the first place, if you invest all of your cleverness in writing the code, then you are by definition not smart enough to debug it”.
“Why instead do we have catfights over mediocre talent in Kendall Square and Silicon Valley?”
I don’t know, but as someone with very mediocre programming talents (for a programmer, for an architect they are EXCELLENT programming talents) I am very glad that there are catfights over me.
Phil,
If someone’s resume said that they learned how to program on the internet but that they have no programming work experience, would you call them in for an interview? Who do you think their first employer would be?
The thing is managers have been trying to offload coding to cheaper labor for years and putting it mildly, it’s obviously not a panacea. Thanks to management completely disregarding my advice on offshore oversight and code review, I got to see firsthand some Chinese handiworkonly after the project was “code complete”. Hundreds of lines of Java constructing SQL statements with string concatenation. (Umm… SQL injection, anyone)? Hard-coded class names in strings with Spring (umm… dependency injection I guess??) Bizarre linked server to another database on the same SQL machine (I guess they didn’t know how to reference a different database in T-SQL?). One stored procedure with @WhereClause param and another one with just @AdhocSQL param. (Why only have one SQL injection when you can have two?)
It turns out that building high quality software is not so easy. Learning how to program is easy. But building a robust, scalable, secure, performant product that meets the requirements, is intuitive, maintainable, and fully tested is not so easy.
With what I’ve seen in the industry, I would guess California’s mess is more related to project management issues. If you can nail down the proper stake holders, clearly define the RIGHT requirements, prevent scope creep, and set a reasonable budget and schedule, it’s not too hard to get a bunch of average programmers to hammer out a so-so product. Of course, in government and politics, those are the things that are nigh impossible.
The market for “programmers” is not transparent enough. Everything from designing a simple HTML page to building low-level kernel debuggers is called “programming”. It’s like putting the qualification “health care worker” on everything from a nurse to a brain surgeon. And most of the people doing the hiring have no way of assessing the quality of a programmer.
As one of those who are the target of this question, what I see is that there’s a huge bias for people who are already in the US with regards to jobs. I have to go through hundreds of job postings to find one that will even consider telecommuting (I had to change jobs recently.) I have over 25 years of experience in programming and I speak / write English very well (I’ve only worked for US companies for the last 15 years) and yet, it was incredibly scary to find a new job.
“Why wouldn’t the millions of unemployed Americans train themselves to code? Why wouldn’t middle class people in China, India, and Africa?”
To me these questions sound very surprising coming from somebody with Phil’s background. Also, they remind me a lot of the typical management response in the middle of a crisis – let’s throw more resources (workers) at the problem. This type of looking at a problem sounds simplistic to me.
A few observations after spending a number of years in the business of software engineering:
1) Learning how to program is easy enough that it can be learned at a very early age. I learned a few programming languages while in high school and neither myself nor my school mates who did the same thought it was a big deal.
2) Being a good software engineer and knowing one or several programming languages are two very different things.
3) The employers are willing to pay very good money for competent software engineers. If they didn’t, my job and that of my US-based team of engineers would have been moved a long time ago to a lower cost center. My employer has development centers around the world so this would have been a very simple change for them. However, the managers in charge of this decision are definitely not stupid – they’ve seen that the overseas teams also have a limited pool of good software engineers and they realize the implications of assigning a project to somebody who simply can’t handle it.
4) Having an organized mind, good work ethics and superior analytical skills is different from wanting to do a job that involves more social interaction. It is a well known fact that many software engineers have poor social skills – how else would you be able to spend up to 10-12 hours a day in front of a display and a keyboard, year after year, and actually enjoy this?! So I believe that actually there aren’t too many potentially good software engineers that are simply choosing other professions.
5) “Why are the Greeks, Portuguese, and Spaniards going bankrupt instead of learning to administer Cisco and Linux?”
I still have fresh in my memory a conversation I had with a nice middle-aged Thai lady who was serving food in a Thai restaurant. She had learned how to manage Cisco routers (her college education from Thailand was in French literature) and she was looking for a job in this domain but couldn’t find one. I admired her a lot for trying to do something to lift herself up but I was quite skeptical about her prospects. I believe the issue was that it’s not easy to find opportunities, even if you’re very well qualified.
Interesting countries as examples, and a well-written post. But hasn’t this conversation already been covered countless times in articles about outsourcing? It’s not the quality of your code, it’s how well your needs are understood and met. It’s not about building the best algorithm if that won’t help businesses make money.
I would point first to web development, FYI. Tell everyone to learn HTML5 and CSS3, plus one language of their choice. The hard part, though, is it’s easily a few years’ work to build up enough skill to know you were doing it wrong (or at least not as well as it could have been done) enough to actually write “good code”.
It could take a lifetime to get requirements right 😉
I am unsure how you contrived this reality because it’s just not true. People in India and China are learning and taking jobs overseas? Not a day goes by that somewhere in the Ukraine, Russia, or China our servers are hacked.
I think you are sorely mistaken my friend.
For what it’s worth- I’m an american programmer and would be happy to get $100/hour paying jobs.
+1 sam, your physics friend aside, to be good is too hard, requires too much raw horsepower, working memory, ability to abstract. plenty of reasonably smart people simply cannot do it.
“Why wouldn’t the millions of unemployed Americans train themselves to code?”
Most likely because of the following reasons:
1. Perception that it is difficult to learn.
2. Sounds boring
3. Unaware of the financial gain and security
I’ve tried to convince friends that were struggling and didn’t know what they wanted to do in life, but I think they are in that position in the first place because they have no ambition.
Because there’s a longstanding view that since programmers love what they do, they shouldn’t be compensated as highly as other fields with demanding requirements. Employees at large tech firms are as highly profitable as bankers, yet they get paid a fraction as much compensation.
So the pool of great programmers is generally limited to those who love the job _so much_ that’s they’re willing to forgo income to do it. Someone with a “125 IQ” and strong math abilities could get a job on Wall Street, or as an actuary, etc and make more money for less frustration.
At Amazon, they target 75th percentile comp – but I really doubt they also target hiring 75th percentile developers on average. So they’re basically trying to hire 90th percentile employees for 75th percentile comp. I’d argue the entire industry does that in comparison to non-engineering industries.
The next time a CEO complains they can’t hire enough engineers, ask them how much over market they’re offering.
Learning to develop software takes long years of grueling work for a moderately low status job. People who excel at it would often excel in other areas as well, and the status and pay is better there, so they do.
+1 to Tom too. Although there probably is something to the longstanding view– it definitely seems like a fun job. Maybe the CEOs aren’t consciously trying to screw anybody just reacting to market realities.
The statistics show that half of STEM graduates can’t find a job in STEM and end up doing something else. There are PLENTY of CS grads available for TRAINING. What you really are saying is why is Google hiring up the senior, experienced programmers.
Tech salaries have been flat for years, which renders your thesis null. What your friends really want is good programmers for a much smaller salary than they are worth.
http://articles.washingtonpost.com/2013-04-24/business/38783974_1_stem-more-foreign-workers-epi-study
There is a glut of engineers. I interview many of them. Of the people that get past the phone screen and do a face-to-face interview, I’d extend a job offer to 5-10% of them. Out of the total pool of people who did some form of an interview (phone screen or otherwise), we’re talking about <1% of potential candidates. Many people can write code, if you're okay with average code.
So out of an ever expanding base of people who learn to write code and have the credentials to have their resume screened by a tech company, we're already screening out many people based on credential and on quality, down to <1% of the total pool. Then consider the growth of the number of tech companies/jobs against the total growth of new good engineers.
It's entirely possible that there are a lot of false negatives in that <1% figure and there's amazing talent out there that is undiscovered or undeveloped. However, that's how the current systems works, with employers looking for either: high profile education or high profile companies / previous projects. Some random person out in the middle of no-where? How does this person get noticed when there are already a lot of people pounding on the door who are mediocre? It's an interesting problem to look at.
There are a lot of highly experienced engineers (20+ years), however, this experience tends to be deep experience with more mature technologies (that they helped mature), while many companies are hiring for platforms, frameworks, and languages that are <5 years old. Juxtapose wanting a plug-and-play engineer with little time for a learning curve with the experienced engineers who learned something else over a long period of time, and you'll see that the numbers aren't there. One could ease that problem by hiring experienced engineers first, then letting them learn the new technology while on the job, but that's an expensive and possibly risky thing to do.
Brad, you can get $100/hour in a programming job, if you move to someplace in the states, and do it on a contract (not full-time) basis, without too much trouble.
Salaries are lower outside of the SV / Boston / NYC metro areas.
There isn’t a shortage of “good” software engineers, instead the interview/evaluation process for software engineers is broken. I have been to several interviews lately for senior software engineer roles and most of the time, the companies are looking for low level programmers, but confuse it with software engineers.
The real software engineering is done most of the time at a higher level (CTO, Architects, Managers, etc..) and the “software engineers” aka programmers they’re looking for just have to be able to write code to do the tasks they need.
This is evident in the way that “software engineer” interviews are conducted. They ask questions like “write a function that shuffle a deck of cards,” “write a function that does this, that, etc. on a whiteboard.”
On the shuffle card function, there are several ways to do it. I was recently interviewed and asked to do that. I solved the problem in one way very quickly, but the people interviewing me only knew one way to solve the problem and kept pushing me towards their way of solving the problem when the way I solved it was more efficient and faster.
The problem is that most companies asking to solve whiteboard problems have one way of solving a problem because they probably look it up on google or in a book without really understanding the fact that there may be other better solutions.
In another interview, I was asked to implement an Email Sender that connects to a database and query emails to send every 15 minutes. When I tried to explain things like modularity, scalability, code reusability and all the good stuff, the interviewer only wanted to see if I could write some simple sql code without being able to test it and thinking it’s production ready.
So yeah, the problem isn’t a shortage of “good” software engineers, but a rather broken interview/evaluation process.
Tom K has the answer. I was coding system software on Cyber machines in the 70’s. 10 years later I saw contemporaries in other fields far outstripping my salary. I jumped to management. The pay went up, I was actually noticed by upper management and was soon a CIO. I spent the next 10 years surrounded by C level staff in different industries who thought programmers were interchangeable cogs. Every time they complained about late projects or poor performing projects, they tried to blame “the programming” talent. I asked how they got better talent in other areas. “Pay more” was always the response. They never made the connection. More H1B visas; find more people to go into programming; offshore the work; these were tried, but never poaching for good people. I almost jumped back to programming during the Internet boom (bubble) but had see too much apathy at the top.
Things can change, but on the other hand, if programmers were well paid and/or respected, we probably wouldn’t’ have so many good start-ups.
@Philg:
I don’t know about software engineers. But I will venture a guess: the probability of becoming a good software engineer, is greatly enhanced if you matriculate at an institution where you have contact with good software engineers. Moreover, direct contact is in every way superior to distance contact. This is just a guess…
I’ve been the software industry for 30 years, both technical and business roles.
There is a shortage of software engineers because the average career of a code writing engineer is short – less than 5 years.
Most hands on software work is crappy. Fix an obscure bug in legacy code. Write a data filter to bulk load reference data. Rewrite some commodity functions for the zillionth time because the managers don’t want code that wasn’t invented here.
Getting assignments that lead to career development is hard. In any large engineering team, the politics are rough. Getting the best work is like fighting a pack of wolves fighting over a deer. Only the strongest survive.
The way to survive is to have both top technical skills (especially the skill to learn new technology fast), and have top social skills. To use the wolf analogy, you must be able to bring down the biggest deer, and defend it from the others. Let the others have the bones (i.e., the crappy assignments). They’ll soon starve and leave.
There is a shortage of software engineers because few people can mix the technical and social skills necessary to survive for long.
this is easy. I know programmers who can write code, but don’t know business. And I have seen the websites they build… ugh, the same crap as they built in college. Nearly every college graduate can write. so why don’t they all write books and increase their income?
Because they would all write the same thing.
This is true in programming too. If 10% of the population knew how to code, what kinds of things would they write? +90% of all industrialized countries are literate. So, why do so few have jobs writing?
This is a problem not about being able to write code, but is one about coming up with new ideas or with the right idea to solve a problem. And once you have an idea, then it’s figuring out (more new ideas) how to implement it on hardware so i works. And that, as Sam pointed out above, takes a particular kind of determination.
No modern education system teaches determination or innovation. Children are both innovative and determined naturally and we drill it out of them. If we didn’t drill it out of them they would be just as innovative and determined at 15 or 20 or 25. How do you run a high school trying to manage a group of innovative and determined boys if you hadn’t been able to turn them into fairly docile obedient people by the time their hormones kick in at 15-16?
On top of that, a desire and ability for critical thinking and abstraction are also required for learning to code. Our schools are not so good at teaching abstract and critical thinking skills either. Personally, I’ve always found that scientists, mathematicians, artists, and philosophers are some of the best technical people to work with. Of course, here again, if we had more scientists, mathematicians, artists, and philosophers our society would be better off.
There is a reason giants in tech are college drop outs, because they are innovative and determined and can engage in abstract thought and think critically and they can’t “wait” anymore.
The basic reason there is not glut of coders is that all our cultures train people to wait for things. Most people wait for someone to tell them what to do, they wait to get a job, they watch TV and wait for something to happen.
The skill of getting yourself noticed and hired is unrelated to the skill of programming, and the skill of filtering a huge pool of applicants without false positives or negatives is nonexistent. I think previous responses hinted at this – the market is grossly inefficient, and no one really knows how to fix this.
The status argument doesn’t work where I sit, my friends that are software engineers don’t go around lamenting their low status. In their worlds their status is high and they love their jobs to boot. I would say the best engineers tend to be unconcerned with status (eg Woz), and they do fine monetarily as well.
Folks: Awesome answers above. Thank you! I want to respond to the idea that it would be too hard for an at-home learner/developer in Africa, say, to build a reputation. I don’t think that is true anymore thanks to open-source projects. In less time than it takes to get a college degree, a person could build a strong reputation as a programmer by contributing to international open-source projects that are, as the name suggests, open to anyone competent. I would think an employer would be willing to hire, as a contractor, a person who had made 500 publicly verifiable contributions to various repositories. At one check-in per day, that’s 1.5 years of work to build a reputation. That’s not an unreasonable investment for an 18-year-old interested in a career.
John Klein: I don’t want to burst your academic bubble, but the institutions where one can matriculate (i.e., colleges) are not where the good coders are generally found. College/university is a good place to learn computer science, which is not quite the same thing. But you might be on to something. Certainly coding skills can be developed quickly working in a group of competent people who engage in code review and that is easier to do if you are on the ground in a place such as Silicon Valley where software development is happening in nearly every building.
Folks who say that there are more attractive higher-paying jobs for people with the same characteristics that make for good programmers: Agreed that this explains why America’s best and brightest head for medical school, law school, and Wall Street, but being a medical doctor in a medium-wage country doesn’t pay that well and it is not possible for that person to sell his or her services over the Internet (except maybe for radiologists). So I still think that the 1990s futurist would have been justified in predicting that the world would be dotted with great self-taught programmers doing contract work from their homes, earning first world wages while living in relatively poor countries. And in fact I have known some Filipinos who were very happily earning $30-50/hour doing high quality work (sysadmin, Web development), but there aren’t nearly the quantity that the 1990s futurists were predicting, which is why I wrote the original posting.
To judge by the responses maybe the original posting should have been “Why is it so hard to learn to be a decent programmer?”
Nearly anyone can teach himself to be a so-so programmer, in relatively short time. And then he’ll be out there competing with all the other so-so programmers, including ones in India and China who can live like kings on an income that wouldn’t pay for rent and ramen in the US.
To be an _exceptional_ programmer, which is the prerequisite for “google and startups are fighting over you”, you need to be exceptionally smart (which the overwhelming majority of people aren’t) AND either exceptionally innovative or exceptionally in tune with some specific range of business needs. You need, in other words, to be able to THINK, not just to CODE.
I’d say you’d be surprised about how rare that is nowadays, except that anyone familiar with your prior writings will know that you (that is, Phil Greenspun) _wouldn’t_ be surprised. (See ref: everything you’ve ever written about the disappointingly low quality of everything ever, in particular the annoying lack of forethought which goes into making things.) Most people, however, would probably be surprised.
I used to be shocked by how much employers were willing to pay me for my talents and time. Then I started participating in the process of interviewing candidates for other positions like mine, and began to realize just how much suck there is out there, even among people who (at least according to their resumes) do this for a living.
I’ve been told by Korean colleagues that in Korea IT is considered to be one of the 3 “D”s (http://en.wikipedia.org/wiki/Dirty,_Dangerous_and_Demeaning) and not much distinction is made between IT and Software Engineer, which is why the pay isn’t good and why he’s working in the US as a software engineer. (And I have to agree with the assessment of IT being an unpleasant medium-skill job, at least the way it’s done in most companies.)
As for skill … let’s just say I work at one of the top software engineering companies, the pay+stock+bonus+benefits is well above average (and several times IT pay), and it’s depressing to interview people with advanced degrees from top schools and claims of years of industry experience …
the interviewer only wanted to see if I could write some simple sql code without being able to test it and thinking it’s production ready
Because there really are people out there who cannot FizzBuzz. If you are too much of a snowflake to write a test on the whiteboard, you probably will suck in other ways, too. They need to filter out those losers.
Not that employers are any good at hiring. They want candidates to appear ready-made, like Athena from the foam, and be productive on day one. A group of generalists will nearly always beat a group of specialists, but lots of managers think they are assembling The A-Team, and need the UI guy who knows Rails and CSS5, the DB guy who knows MySQL, the engine guy who knows Java, the QA guy who uses whatever Rational is selling; everyone all in their little roles. And they expect them to be cheap, too.
Interesting responses here.
I just have to say that I’ve been interviewing folks over the past month for 2 manager positions, 2 engineers, and 2 developers and haven’t found anyone worth hiring.
Most of them came from larger companies and had years of experience and couldn’t pass simple abstract non-code tests. Things like “how would you approach designing this application?”. Some of them couldn’t even cough up a good answer to “What do you want to do?”. And these are the folks that pass a phone screen by a recruiter.
It’s a bit perplexing. Everyone I know in the industry at other growing companies are having the exact same problem.
I’d jump at the opportunity to hire someone with no “real” work experience but with a demonstrated commit history on something like OpenStack (or whatever). That shows you can create something *and* get along with a community.
We have something north of 100 positions open that need filling here at Rackspace. We pay market or above with very good benefits. Most of the jobs are here in the wilds of Texas, but talent trumps location. Details at http://rackertalent.com
In a nutshell, HR departments screen applicants the same way teenage girls screen potential boyfriends.
To get an idea of the nature of the problem and it’s effect on the tech hiring market, google the following phrases: “passive candidate”, “list of boyfriend qualities”, “social proof”, “credentialism.”
To understand the scope of the problem, review tech job requirements with a software engineer and you will find that most “required” skills and attributes are contradictory and almost impossible to find in one candidate.
Bypass HR, problem solved.
The New York Times had an interesting article two days ago on a startup which is trying to identify promising candidates by such things as the degree of reuse of their open-sourced code: http://www.nytimes.com/2013/04/28/technology/how-big-data-is-playing-recruiter-for-specialized-workers.html?pagewanted=all&_r=0
“Any good futurist circa 1990 would have predicted that by now most of the world’s programming would be happening in low-to-medium-wage countries and that customers wouldn’t have to pay a lot for high quality software.”
I think there’s actually a fairly low limit on the number of potential US employers that are willing to endure the inefficiency of overseas development. There’s plenty that try and some might even succeed in convincing themselves that they are saving money, but they inevitably suffer from a variety of unintended consequences. Examples being the code quality problems mentioned by others, issues of long-term maintainability by an unstable workforce (lack of knowledge continuity) and the loss of good workers unwilling to put up with inconvenient working hours to facilitate collaboration across large time zone gaps. The issue is much deeper than $$/hour.
I’ve been programming professionally for more than 20 years now. I’ve seen many attempts to offshore my work with every economic crisis. but with all my years of learning, I can do most coding jobs 10x faster and more precise than most that I work with. That is because I follow and try every new tech there is, it is a passion that most people do not have but it is fundamental to be a good developer.
I used to spend 20 hours a day trying to understand how games and software was working, while my friends were out having fun. nowadays most kids are just playing games, they don’t care enough to learn how it all works.
The facts is, developing software requires a lot of skills. Management tend to think that writing software is just typing text, and that anybody can learn. But writing software to handle a fighter jet, trade billions of dollars, control traffic light, tranform signals from an NMR scanner, … etc that is way too complicated and dangerous to go wrong. It requires very specialised people. People who are not in it for the money but for the journey to master all the skills.
David
To put many of these comments in a small capsule: About 1965 I took an American Management Association-sponsored course on the relatively new field of computers for data processing – in Belgium. The instructor-expert said, “I can’t teach anyone to program; you are either born with, or without, the ability to think.” Doesn’t sound like much has changed.
Regarding your question about why Spaniards are not learning to code or administer systems it may be that getting a job in those areas is not be as easy as you may think, at least there.
Being originally from Spain and having a BS and MS in computer engineering (from a university in Spain) I can tell you that it is much easier to find a good/interesting job in that field here in the US (at least in Silicon Valley) than it is there. There are just many more opportunities here. It may be in part because legislation is more business friendly here. In Spain it is more expensive to downsize because by law companies have to compensate the employees that are losing their jobs and, as a result, companies will be more cautious before hiring new workers. It may also be kind of a vicious cycle. Because there aren’t that many qualified programmers there, companies do not thing of it as a good place to start a business in that area. And because the job prospects are not that good not too many people think of pursuing a career in that field.
Also, for the few jobs that are available, the requirements are high and usually include having a university degree. So, a significant time and economic commitment is needed just to meet the basic requirements to qualify as candidate for a job (just having access to a $300 personal computer and an Internet connection wouldn’t be enough). For example, I seem to remember that when I came to the US in 1996 HP would only hire people with a BS degree or equivalent to work at their Barcelona offices. Not only that. Before being considered for an interview, you would have to take a written test administered by some independent company that was providing that service for HP. I took that test once shortly before moving here. I am not sure what position I was applying for. It may have been chip design and not programming. But whatever the position was, the test included questions not only about the subject matter but also about control theory (not just a few questions but a lot of them). I guess that area of knowledge must have been useful/needed for the kind of products that are developed there (large format printers, I believe). Anyway, my point is that for the jobs that are available there (and maybe because there are many more candidates than jobs) the requirements are high and not exactly easy to meet.
I just took a look at what jobs are available at monster.es and the first one I looked at was actually for a position in Belgium even though the summary description listed Barcelona as the location. So, the ability/willingness to relocate was another requirement. And a very high level of English was also mentioned in the description. For one person that doesn’t have these requirements and skills it would easily take 5 years of complete dedication plus a significant amount of money to get to the point where the requirements listed are met. And he or she would still have to compete against other candidates in order to actually get the job.
Yet another failure of market theory to predict human behavior and yet, people still insist on building policy and practice upon such shaky foundations only to be surprised, once again, when cracks open and things collapse.
It is indeed a mystery.
As to the specific question of why there isn’t a glut of good software engineers, I think the answer is that you are asking the wrong question. Forget about software engineers, or programmers, they are a distraction or it seems that way to me. The question I think you are really asking is: why is it that, for the subset of problems that can be addressed with computing and software, so few of them are successfully addressed by computing and software? I’m not convinced a glut of good software engineers would significantly alter this state of affairs, because I think the real bottleneck lies elsewhere.
A good software engineer is worthless without sufficient knowledge of the problem domain. Some might argue that a good software engineer should possesses the skills and inclination to obtain sufficient knowledge of the problem domain, but my experience is that many otherwise excellent software engineers are lacking in one or both of these critical qualities, and those with all the requisite skills still have serious learning curve to climb to acquire sufficient knowledge of non-trivial problems.
A day or two ago, I was on a site where someone was asking for more information about “Script Docter,” some in-house software that Pixar used to prototype movies. Someone helpfully replied that the guy who wrote “Script Docter” has given a lunchtime talk on the software at WWDC for the past few years. The commenter then went on to say that it was software that any competent Cocoa developer could write in a week — That may be true, but it ignored the fact that the developer had been working at Pixar on specialized productivity tools for almost two decades. The number of Cocoa developers who could write such a tool in a week, or even a month, is probably quite small. Moreover, I think that the number of developers in China, India, Portugal, Spain, Greece and the Philippines who could have written Script Docter in a reasonable amount of time may well be 0.
Of course, this situation doesn’t just come into play with Pixar’s problems. It also comes into play with many other problems with try to apply software to. It takes a great deal of communication to get software built to address non-trivial problems. Domain expertise must be communicated. Cultural and language differences can mean even more information needs to be communicated, while at the same time, making that communication more difficult.
A glut of software engineers could change things, but it would have to be such an oversupply that those engineers ended up flooding the overall job market. I think we are a long way from that happening, too many of our high schools students never graduate, and of those that do, far too many lack sufficient literacy, math and critical thinking skills. The system has a long way to go before it can close those gaps AND add basic programming competency to the mix.
I think the real problems may be even more fundamental though. Software development is dismal. It seems that every decade or so, everything gets thrown out and replaced by people who know almost nothing about what came before them (and that which doesn’t get thrown out, fails to adapt and stagnates). They can get away with this shit because a decade’s worth of Moore’s law has boosted available resources and expanded the market (by lowering costs). The only hope of things getting better will be when either Moore’s law tapers off and/or the world is flooded with cheap, efficient hardware. Until then, we’ll be building on the toes of giants, instead of their shoulders.
+1 to long time programmer, I am really still at the start of my software development career and you are absolutely right, I have only been able to develop by securing a project of mine own and working on it intensely. I am very enthusiastic, always reading and partcipating in the community but my growth as a developer is determined by the work I do during my 9-5 hours.
At home I enjoy what I do and spend as much free time as possible programming but there is only so much time I can spend out of work due to social commitments. I very much believe my growth is based on 20% academics and 80% experience.
Would like to hear more on your views about this!
“…most of the world’s programming would be happening in low-to-medium-wage countries and that customers wouldn’t have to pay a lot for high quality software…”
Indian IT is booming (and driving pay rates down in the West). But the quality is very variable, project coordination is difficult and generally it doesn’t work out any cheaper. Even so, projects continue to be outsourced because management is sold on your line.
So why isn’t there a glut of good software engineers? ‘Cos the pay and conditions are lousy compared with law, medicine, banking. Talent can choose and not everyone is dumb enough to choose IT.
Like any profession, there are only so many people in a given population who will be good at it. Today, many of the engineers I interview got the degree “because it’s good money”. When I got into engineering, it was because I loved it. I really didn’t think about how much money I’d make.
“Programming is easier to learn than a foreign language”
I dispute that. It’s certainly different – but you can learn to speak a foreign language well enough to get by in a year or two taking night classes, and you learn it out of necessity once you are immersed in a place where you have no choice to speak it. Unless you’re actually teaching a language and have some need to understand every last nuance deeply, you can learn it well enough for most purposes (e.g. to communicate with others who speak that language) in a relatively short time.
On the other hand it takes years and years of active experience to become even a “pretty good” programmer, and you never get to catch your breath — you never get “good enough”. The technologies we use change constantly – we have to go out of our way to stay abreast of new tech, tecniques, trends, and so on. A novice foreign language speaker can get by with no ill consequences — as long as he doesn’t start an international war, he’s probably OK making a lot of mistakes. Programmers, on the other hand, don’t have that luxury.
20+ years into my career as a programmer, I feel like the areas that I don’t understand just keep growing every year, rather than shrinking, despite constantly making an effort to learn. This is the nature of the game. New tech arises. We have to decide what languages and technologies to embrace. We can spend huge amounts of effort learning something that becomes obsolete. It’s exciting, frustrating, thrilling.
But it’s also incredibly hard. Foreign languages don’t evolve, don’t have a huge associated cottage set of tools, frameworks, and related technology that constantly change and you must also be fluent in in order to be useful.
Just look at any job listing. The requirements could be like: “Must be skilled in [some specific] SQL, C, C++, C#, ASP.NET, Visual Basic, HTML, CSS, Javascript, Perl, PHP, jQuery, Node.js, [and add a half-dozen other niche tools used by the organization]…..”
Almost all of those technologies are extensive, require years of hands-on experience to understand deeply, and constantly change. And that’s completely typical.
If anyone asked me if they should become a programmer, I would advise them: only if you love every last second of it.
The idea that anyone can be “trained” to be a good (or really good) software developer is seriously flawed. You can train someone to “write code” but you can’t train them to think the way good software developers think.
Also, there butt-loads of good software developers throughout the US. They are spread out in every semi-major city in every state. They work at all kinds of places. When you interview people for a job you are not necessarily going to get all of the best and brightest from all over the country – you will get a small subset of people who are in your area, have a certain skill set, are looking for a job, think your company looks interesting, etc.
“Why are the Greeks, Portuguese, and Spaniards going bankrupt instead of learning to administer Cisco and Linux?”
Portuguese national here.
Long story short, the general feeling is that a good software engineer isn’t much different from being a mediocre software engineer because companies prefer to employ 1 or 2 code monkeys (and pay the least possible) instead of a better, more expensive software engineer.
I believe this happens because Portugal’s IT market only cares about two things: either accounting software or consulting services for ISPs. R&D is either limited to the academia – completely cut-off from the market – or to specialized centers inside the large ISPs, so why pay two or three times more for a better engineer if you’re not going to need his expertise?
The idea is to pay as less per person as possible and still demand they do overtime because “the market is bad” and if you don’t like the conditions too bad, someone will. Outside IT it’s even worst, as some companies are offering a ridiculous 300 $ per month (that’s 200 $ below the minimum wage!!) and they even have to nerve to tell the candidate that it’s their way or the highway.
Screw motivation, get back to work grunt.
Wages have been getting smaller and smaller, while taxes (for both employees and companies) skyrocket and the job you do each days becomes more and more pointless.
Wanna get rich in Portugal? Become a mayor and start embezzeling. There’s this one guy that’s currently in jail for embezzlement and he still has the balls to claim that he won’t renounce his position because he can work inside jail.
/rant
I think the fundamental problem is we don’t have realistic image of “programming.” Fundamentally software development is a creative process yet we generally regard programming as being the realm of purely logically minded people, and not the realm of the creative types. In my opinion, it’s more art than science. Lots of people can code (that’s the science part), but in my experience relatively few can actually do what I consider real programming (the art).
And the “art” part varies, too. I frequently liken programming to writing – if you give a specific story outline to Steven King and Jackie Collins, each can give you book that covers all of the items in the outline, but they will be very, very different reads. It’s the same with programmers (especially once you get below the user interface and look at the source code).
Speaking of writing, we’ve had typewriters for years (and pen & paper for hundreds of years before that) – why don’t we have more terrific writers? Shouldn’t we be awash with that kind of talent now, too? (That’s a rhetorical question, by the way – I know there are many tremendously gifted and capable writers out there. The real question is are there more of them than there are gifted and capable programmers?)
To be a really good programmer you have to have some traits same way a detective or scientist has. You have to predict as many of possible outcomes as possible even impossible ones (which tend to happen more often than we usually expect) and trust no one in the code even to yourself (defensive programming style). That traits will give you more stable code that can work even in mission critical devices or services.
“Why are the Greeks, Portuguese, and Spaniards going bankrupt instead of learning to administer Cisco and Linux?”
Portuguese national here.
Long story short, the general feeling is that a good software engineer isn’t much different from being a mediocre software engineer because companies prefer to employ 1 or 2 code monkeys (and pay the least possible) instead of a better, more expensive software engineer.
I believe this happens because Portugal’s IT market only cares about two things: either accounting software or consulting services for ISPs. R&D is either limited to the academia – completely cut-off from the market – or to specialized centers inside the large ISPs, so why pay two or three times more for a better engineer if you’re not going to need his expertise?
The idea is to pay as less per person as possible and still demand they do overtime because “the market is bad” and if you don’t like the conditions too bad, someone will. Outside IT it’s even worst, as some companies are offering a ridiculous 300 $ per month (that’s 200 $ below the minimum wage!!) and they even have to nerve to tell the candidate that it’s their way or the highway.
Screw motivation, get back to work grunt.
PS: mods, why did you delete the previous post … ?
Intellectual property rights are not well respected globally. Every thief out there, most don’t realize they are a thief, contributes to the culture such that investing (always a long range activity) in software is too hazardous or even impossible.
The U.S. is a leader in defending intellectual property, therefore they are the sweet spot for software development.
Chad hits a good point: if I have my software built by a Chinese company, it’s very likely that, if it works, they’re going to sell it to someone else, too.
It’s possible to work around this by planning for it to be open-source, but then what are *you* doing? You surely don’t know the code as well as the Chinese people writing it.
Speaking of open-source, that didn’t put software developers out of jobs, although a very simplistic view of code would say that was inevitable. The thing is, there is not a finite amount of software to be written. If someone writes a bunch of code for free, there is *still* a lot more gains to be gotten on top of that.
As software developer here in Portugal i don’t know what is the reality in the US, but in Portugal we also have a lack of people to work in the industry, and the main reasons are that :
a) The young people don’t want to learn the skill set to do the job, because they know they are going to be ostracized by others;
b) The pay is low, $30/$50 would be very nice;
c) Because the pay is low many IT workers go to Germany, Uk, France even Spain;
A good software engineer is someone who can solve problems, someone who can see past the lines of code and see the many other variables in play, and someone who can communicate in plain English about how a system works in and around your product.
When I started 20+ years ago I knew C/C++ down to earth. Today, I code in 5 languages and don’t call myself an expert in any of them. However, I’m regarded by my organization as indispensable engineer because I’m a problem solver.
When I interview new candidates, I care less about their domain, or programming language experience. What I want is someone who knows how to find answers, and someone who sees past the given use-case or logical flow in the code.
I am stunned by the tone of some of these posts. ‘IQ of > 125’, ‘mediocre’, ‘code monkeys’, ‘the best and the brightest’. Look around, guys, really look at what you do, day to day.
Programmers are not superior beings, able to gaze at their navels, hum a mantra, and then spout perfect or even good code.
The variables in developing robust, viable software systems are enormous. The challenges vary from mind-numblingly boring to moments of ‘oh my God, how cool it that’ and everything inbetween.
‘Good’ programmers have one common trait; they care about their work. Everything else is learned, easily or painfully hard, it doesn’t matter.
The fact that the market hasn’t responded with higher salaries reflects that fact that making money on software is a crap-shoot.
The economics of software are different from anything that preceded it. Software does not just manifest the insights of the engineer – it propagates them at almost zero cost. This means that one great software engineer at a competently managed company can wipe out a whole industry of less competent practitioners. This is why we see cannabalism in our hiring practices: senior engineers develop platforms, and then are replaced by the junior developers they empower.
What this also means is that the great developers, if they have domain and business savvy (or link themselves to someone with that savvy) can make a pile of money.
Well – of course – they can make a pile of money until hungry developers looking for an opportunity copy the superficial functionality and offer it to the world for free, given that they can’t make any money in direct competition. Then for the original developer it becomes a hard battle of extension, integration and refactoring to maintain a competitive posture in the marketplace.
So why pay a lot of money to produce something when the risks are so enormous?
There isn’t a glut of good software engineers because the problem isn’t learning the programming languages. Basic grammar is something most people in developed countries have been taught before they hit puberty but there are not many high schoolers would be considered on par with the Bard. Because mastering grammar is not the same as mastering writing. The same is true in programming. Just because someone can use an if/else does not mean they are capable of writing a large application or even a small one.
Good programming starts with logical thinking, a skill in short supply. Once the problem has been analyzed, devising a solution that solves the problem without a bunch of excess cruft around it. Then the slinging of code starts.
I am a self taught developer and after a couple years of learning I was hired at the upper end of the pay scale discussed in the article. I work with a number of developers who have been “professional” programmers for 5-15 years who would not be considered good by any definition of the word. They know the grammar but they struggle to write code that is clean, efficient and effective or in other words not awful.
The supposition that the world will be glutted with good programmers has a presupposition that the world will be glutted with intelligent people, as though by magic. Not going to happen…
I reckon it depends on the person, but in my case, I found learning German well (I grew up speaking and hearing English only) was, although not easy, quite a bit easier than learning to program well. Mandarin Chinese I realize is a harder language for English speakers to learn than German is. But just as there are big differences between the difficulty of programming languages (Basic and C++, for instance), there are large differences in the difficulty of human/natural languages, too.
There are several reasons (IMHO).
1 – Good software engineers are often overlooked because of age. Many companies have no interest in quality software engineers when they get near 50.
2 – Cheap, low quality, non-American software engineers help the short-term financial statements, but not the long-term health of the company.
3 – The myth persists that non-American software engineers are smarter and do better work for a cheaper price. The cheaper price is true, but those of us who have dealt with offshoring and H-1B visa holders who work cheap know the quality is rarely there, and the ability to figure out problems that don’t fit the textbook is rarely there, either.
4 – American education, at least in government schools, has been dumbed down the last 30 or more years. Today’s 20 and 30 somethings are more likely to have been indoctrinated, not educated. The basic knowledge set of a 1970s high school or college graduate is generally superior to that of such graduates in the 2000s and later. That has led to a smaller pool of people with the character, critical/logical/reasoned thinking necessary to be a good software engineer.
The fact that the market hasn’t responded with higher salaries reflects that fact that making money on software is a crap-shoot.
1. Salaries have gone up in certain localities, like SV. If you want to live there, you can make $100K out of college.
Now, the better question is why salaries haven’t gone up as much in other areas.
2. Software companies make boatloads of money. Here:
http://www.techspot.com/news/43846-apple-overtakes-google-becomes-first-in-yearly-profit-per-employee.html
They aren’t quite in Goldman Sachs territory, but Goldman Sachs will proudly give all that profit to the employee instead of the shareholder. If software engineers can manage to get better at negotiating, they too could have $200K – $300K salaries.
Power to the proletariat!
I would guess that outsourcing nations at least have a culture which discourages and suppresses talented engineers. The IT industry in India and China is built largely on providing cheap software to the kind of business that looks at software as a cost center to be minimized, rather than as a product of variable quality. Therefore, anyone who really does feel passionate about the work either gives up or moves to North America. Most people are likely so grateful to have a good job that they don’t think twice about the lack of attention to quality.
I graduated with a CS degree. Big companies pass on. Tons of resume without any answer from them. I have to settle for small ones which pay so little. What is the chance of self taught programmer get a job? very little.
There was that comment in “snow crash” about how (once everything that could possibly be outsourced from the US has been) the US is left with “music, movies, microcode, and high speed pizza delivery.” Three of the four make sense, but I’m not sure why microcode (he clearly means software) would be in here. Maybe there’s some social paradox where top coders can only be drawn from a social environment that actively (perhaps unconsciously) discourages people from getting good at programming.
I can teach a Greek to speak English, but that doesn’t make him William freakin’ Shakespeare. I can teach a clerk to write Excel macros, but that doesn’t make her Ada Lovelace. There are more programmers than there are good programmers, and more novice programmers than there are experienced ones.
A four year Computer Science degree is about 90 credit hours of specialization. It takes 2 full-time years (on top of college) to acquire a journeyman’s competency with C++. Your 10-year-old’s six months of python does not make him employable.
There is no shortage of talented programmers. There is only a shortage of talented US programmers with a CS degree and 10+ years professional experience willing to work for $35/hr. All shortages can be cured with better money, more perks, and greater respect. There are sometimes surpluses of programmers, like during the Great Recession when some companies were laying off and nobody was hiring. That there are never shortages but there are sometimes surpluses is a factor in students’ willingness to blow 4 years getting a degree, when they can get other jobs with less up-front effort.
Google and Facebook have revenue streams so wide they can afford to pay top dollar and awesome perks for programming talent. That’s why they get the talent. If startups offered great wages instead of a dream of wealth and a year living on ramen, then Google and Facebook wouldn’t get all the top talent. Plus if you’d worked with this top talent, you might be unimpressed with what they get for their money.
Folks: I don’t think it is apt to compare programming to being the author of a novel or a play (see above comment above Shakespeare!). A programmer who simply listens carefully to user requirements and then implements a system to meet those requirements will be considered at least a “good” programmer. He or she need not create an entire world the way that an Edith Wharton or Tolstoy might. A better comparison, in my opinion, would be to journalism. And most people do accept that a capable journalist can be produced by training.
Philip: Elsewhere you’ve said that a lot of computer science university graduates don’t learn how to be an engineer, but how to work for an engineer. Is it fair to think that someone who self-studies programming through books and free online resources would be capable of deciphering user requirements and building a software system that meets them, when formally-trained computer scientists seem unable to?
Why are the Greeks, Portuguese, and Spaniards going bankrupt instead of learning to administer Cisco and Linux?
“We’ve all heard that a million monkeys banging on a million typewriters will eventually reproduce the entire works of Shakespeare. Now, thanks to the Internet, we know this is not true.” – Prof. Robert Silensky
That’s why.. :))
@Jeff Jones
“The myth persists that non-American software engineers are smarter and do better work for a cheaper price. The cheaper price is true, but those of us who have dealt with offshoring and H-1B visa holders who work cheap know the quality is rarely there, and the ability to figure out problems that don’t fit the textbook is rarely there, either”
I’m not sure who believes that non-American sw engineers are better and cheaper, but I can share a couple of personal stories as a proud former H1-B visa holder and, now, as an even prouder American citizen. More than half my colleagues at my various sw engineering jobs in US were born outside the country. The proportion of good or even outstanding engineers was the same among the immigrant and American groups of people. My compensation package has been competitive.
My experience with the offshore teams has been somewhat less encouraging. With the exception of one country, where the company I work for has had a presence for a long time, the other remote sites, which were newer, had a lot of junior engineers. Among these junior engineers there were a few who were very good and a couple who were outstanding but the problem was that the attrition was high and whoever was a good engineer could quickly find a job somewhere else, with better pay. The most recent example includes a person who was hired by Facebook and relocated to SV (and yes, the compensation package was competitive and it fit that person’s experience). Another issue was that even for those who stayed with the company for a longer period of time it wasn’t easy to advance in their careers due to a lack of good projects – the older teams, which were usually based in US, had priority, were closer to the management, sometimes closer to the customers etc.
Really great discussion here, and relevant to me because I’ve just a fair of amount over the last week interviewing intern candidates here in Austin.
My background is working a software developer in a number of small companies here in Austin since I retired from the military in 1994. I’ve been at present gig for ten years, and was the day 1 employee.
It became clear to me very quickly that in the civilian/commercial world hiring the right people is critical to the success of a software company. For that reason I’ve always stayed involved in recruiting as much as I can.
I mention this because I don’t think at all like a big organization HR person, I’m probably an outlier.
I’ve never liked the term ‘software engineer’, and only a handful of people can legally use that term here in Texas, as ‘engineers’ are required to be licensed.
I strongly prefer ‘software developer’ because all my software work has been for the purpose of delivering a commercial software product to our management for them to sell.
There’s a lot more to product development than just ‘programming’, although programming is the core task. I wish colleges did a better job of teaching their students how to deliver a finished product that another person could use.
Psb aks:
“If someone’s resume said that they learned how to program on the internet but that they have no programming work experience, would you call them in for an interview?”
In my case, yes, absolutely, if they can show that they’ve written software that others are using. An application in the Apple App Store is the gold standard, not because Apple products are that special, rather because those apps are curated and will meet a known quality standard. (The demand for iOS programmers here in Austin is huge. Anyone with an non-trivial app in the App Store can get a job inside of about three days here, nobody will care about their actual background.) In our case a Google Playstore app or recognized contributions to a significant open source project would also justify at phone screen and probably an in person interview. That’s as least as good to me as a CS degree with no practical experience.
In the case of entry level technical degree graduates I want to see some sort of non-coursework self initiated software development project. Often it’s a game, I’ve seen pretty decent chat applications and interesting web applications from the more motivated students. The better the application, the less I care about their gpa.
I’m not interesting a recent graduate with no self-motivated work, no matter how high the gpa.
The thing is this. Lots of people major in CS and even work in the field because its a good paying job that is inside, air conditioned, with little or no heavy lifting. Or because their parents told them to major in CS or EE for purely economic reasons.
If you don’t actually like to develop software, you’ll never be good at it IMHO. Most people simply lack the mental configuration that motivates good developers. That’s why it’ll always be hard to find good software developers.
This concept also applies to welders.
One last thought. Every good software developer (or ‘software engineer’, or ‘architect’) is a good programmer.
That’s why we start every software development interview with a fizz buzz type question, and then ask increasing difficult questions until we feel like we’ve reached the candidate’s comfort level.
We understand that many good developers can’t just jump up to a whiteboard and write perfect code to reverse a linked list with no notice in an interview situation. But at least they ought to be able to make credible effort.
Phil, you site open source projects as reputation builders.
Do employees really value open source contributions?
My experience is that management sees them as distractions; things that
can steal valuable company time;
Other indications are that open source people are seen as not pliant
enough; and people with too strong opinions of their own are not valued
int the market.
In general I don’t know if I should hide my open source projects on the CV or not.
On a practical note; I think the open source movement comes with its culture of sharing and its own set of ideas; the whole thing came out of the Hippy movement (just look at the long beard of Mr. Stallman 😉
Now with all things cultural there comes the same problem: it is hard to transplant a given culture into the third world; all third world countries come with their own culture, with their own system of values. Western countries can’t be simply transplanted into the third world, it can be adopted, but the process takes time.
Exceptional programmers are those that have
1. Incredible high IQ
2. Determination to learn.
While you can blame the lack of glut of good software engineers on the lack of determination (there are a lot of programmers who treat their job as any other a 9-to-5 job) the high IQ is actually the big problem. I have an IQ of 130 and, looking at the source code of some open source project, I feel stupid (not because I don’t understand the code but by the elegance of those solutions).
http://www.iqcomparisonsite.com/iqtable.aspx shows how few people are candidates to become good programmers. Obviously there are people with high IQ that are not interested in programming so the pool shrinks even more.
There are many more good programmers, but toxic is the ground that has developed in the industry for allowing the flowers of intellect and creativity to blossom. Let me get back to that but first allow me to disrupt the topic:
Why interview? Why cull resumes? Both are LOW INFORMATION. They are very low information. Few humans except newborns with cute puffy faces and babyfat don’t show well in either. But babies can’t code or engineer useful systems or attack human misery using technology.
Stop interviewing. Dump all the resumes. Fire more often. Hire more often. Somebody who comes in, they want to see you and your work just as much as you want to see how they fit in that garden of delights known as a productive workplace and how they can bloom and with that bloom add to the beauty of your garden.
So let them actually work for a day, a week with you. Come in the door and follow, as an extra hand, extra eye, extra mind at the shoulder of a key worker. See how it goes. Fire them! Or hire them. Either way a far far better hire than all the lying resumes and all the bipolar social phobic or social gadfly personified interviews.
The best chefs do this. A sous chef walks into a kitchen, the lead chef says “Follow me!” and that means follow the lead and pitch in. A chef looking for a new position quickly finds one that is compatible with his skills and personality. Many trials, but REAL trials. As we who follow the Food and Cooking shows know, great chefs are not unlike great programmers. A creative and cookbook mix, the clerical mundane processes of programming and engineering are matched by all the rigorous prep work and maintenance of larder and fridge that a master of a kitchen must ever continue, and yet no good restaurant survives without a chef staff committed to lifetime learning and innovation.
I recommend reading ‘I am a Bad Programmer’ on Hacker News for relevant perspective. It recounts the travails of an experienced programmer who cannot get past the interview stage eventhough he desperately needs a job. If it’s no longer on the first page, you can use their search box to find it.
Seems like there’s two groups posting in these comments–the disenfranchised and the “enfranchised”. The disenfranchised developers describe a system with arbitrary HR departments and companies hiring for quantity over quality. The “enfranchised” founders/recruiters and high-end programmers say it’s very hard to recruit someone with high enough quality to develop challenging products and that Silicon Valley startups pay well and still have trouble finding staff.
As a technical founder at a challenging startup who is deeply involved in recruiting; for local talent, we expect to pay $90k for a college graduate showing some initiative and $110-200k for senior developers…and it’s very hard to find the right people (I sympathize with the Rackspace gentleman). For remote developers, I’m currently paying about $90k for senior developers (and they probably have a higher quality of living than our local senior developers due to lower costs of living (I’m unlikely to consider an inexperienced person because our product is sufficiently challenging and the coordination overhead makes it unfeasible…other companies may differ). Developers who maintain an open source project relevant to our product are clearly desirable and our current tech lead was found that way–it demonstrates both skill and an understanding of what’s useful (they anticipated what we’d be looking for by developing and maintaining a project we later find useful).
For projects such as my company’s which can dramatically benefit from exceptional programmers (and many Silicon Valley companies are similar) it remains hard to find good staff, local or remote.
Yes, recruiting is a deeply flawed process, but that’s the nature of the beast. Also, I find most developers incredibly passive. If finding a great job is so important, why would they not develop a portfolio of (open source) code they can show to a potential employer; why would they not look into themselves to identify technologies and products with which they’re passionate and directly contact relevant companies. Instead, they spam companies without regard for goodness of fit (rarely do I receive a cover letter explaining why a candidate’s skills/interests match our company’s and although we have a publicly available, free product, rarely do I see a candidate try it out) and many use recruiters who will admit to not even superficially understanding the relevant technologies.
I think there’s a bifurcation between commodity software companies and the commodity developers who work for them…and high-skill companies and the high-skill developers who work for them. We sometimes have to interact with the former companies and although the job titles and CV’s might look similar, they’re worlds apart in ability to get things done. Exceptional developers know many technologies and easily pick up new ones, so high-skill companies don’t require an exact fit (e.g. a developer just needs to know any SQL variant or any JavaScript framework…not our specific variant/framework). Yes, this is implying that “if you’re whining, you’re probably a commodity developer”, but I haven’t met any exceptional developers who have trouble getting work while I haven’t met any high-skill companies who can easily hire for open positions.
Last week I had to conduct a job interview at my company, which is a very large corporation. We were provided with the corporate instructions for conducting interviews. The interview must be 45 minutes in length. Greet the candidate, explain the interview process, etc.
30 minutes of the interview must be spent asking the candidate questions to determine whether he/she is a team player, communicates well, has initiative, is good at problem solving etc. Tell me about the most effective presentation you ever gave. Tell me about a time when you had difficulty communicating with a co-worker, and how you resolved that problem. Tell me about a time when you had an idea that was rejected, and how you eventually got the idea accepted. Tell me about a time when you helped a co-worker learn something. Tell me about something you learned on your own initiative.
No time need be spent determining if the candidate can actually write a program.
Interesting point.
As the Marketing Manager for a small business, we face difficult decisions when hiring freelance software engineers for our cloud-based recruitment software.
However the trend that we’re witnessing in the UK is that the cost of acquiring the talent is dropping. The market has become saturated to a certain degree, with our universities churning out capable graduates hand-over-fist.
I’d point to Mark Lutton’s point about the depth and relevance of the contemporary interview process. Perhaps too much time is being spent on ‘culture fit’ rather than capabilities.