Multicore to Many-Core: Hardware too far ahead of software?

from Tom's Hardware (tomshardware.com)

Following my last post on “Keep talking about Multicore” a few colleagues pointed out that discussing Many-Core is applicable too. I certainly don’t disagree since Many-Core processors are coming as TNBT (The Next Big Thing). I think we will see adoption into specialized markets sooner than others. In particular networking which is already comfortable with multicore will use the many core tiles to speed up specialized processing. I can see applications in other areas such as imaging and devices that need higher end processing of audio, video or graphics.

In general, as I stated in my previous post, the embedded market can be slow to adopt new technology in specific market verticals – not because they are old school but because the product timelines and planning are years in advance and product life cycles are much longer. The move to multicore is underway and many-core in the near-term will be more niche than commonplace. Why? I think because we are still coming to grips with the software complexity of multicore. Moreover, tools are still catching up to multicore, how will we debug a 100 core system? Many-core will likely be handled the way heterogenous multicore solutions are today (e.g. general purpose CPU plus DSP or GPU). Vendors might supply libraries that take advantage of many cores will leaving a small number for general purpose processing. I have heard customers say that, heterogenous systems are particulary hard to debug because they are usually supplied with two disparate tool chains. Interestingly, an article from 2007 was quite prescient about this very topic, supporting my premise that hardware has leapfrogged the software – of course, would I quote an article that didn’t?

In the near term, I think we will see 16 to 32 core chips which will be “quite-a-few-core” systems. In these cases, I think a hypervisor solution can bring sanity to the solution by allowing you to create several virtual targets in one. For example, you could create a four core controller plus 12 specialized processing engines (this is being done today for deep packet inspection). This is manageable because the application complexity is isolated to your four-core target and the specialized processing engines are identical and relatively simple.

Whatever way the many-core technology pans out, I still contend that software is behind the curve versus the hardware. And I still won’t stop talking about multicore…

Don’t Stop Talking about Multicore

I keep telling anyone who will listen (and some who don’t really want to) that product managers and marketers shouldn’t stop talking about multicore. Of course, this is specific to one’s own market but if you are in the embedded business these are still the early days. The embedded market is really a vast mixture of different types of products and applications and the applicability of multicore chipsets is different for each. In networking applications, multicore has come and is already designed in. In fact, these companies are on 2nd and 3rd generation chips with 8 to 16 cores. Industrial and medical companies are following but likely to adopt multicore chips in the coming 2-3 years. Some market segments may take even longer, such as aerospace and defense.

One thing I’ve learned about the embedded market place is that its not easily described and defined. For example in consumer devices the time to market windows is months and things must happen now! In defense systems, lifecycles can be decades and next generation products must be carefully planned to fulfill their expected lifespans. What this means to the marketing department is: don’t stop talking about multicore! Each of your target customers has their own unique adoption cycle and until the majority of the market is on multicore platforms, it’s a bad idea to stop talking to the market about what you have to offer.

Why?

I think multicore is a disruptive technology in computing and its forcing a lot of development teams to rethink the way they do things. Whenever companies are forced to sit up and rethink things, there is opportunity. Opportunity for the companies to improve things like performance, thermal profile, bill of materials costs. For vendors, this is the opportunity to get close to your customer and help them solve the issues they are likely to face and, hopefully, sell them some product along the way.

Software Can Kill, Really

Image from Wired.com

Wired recently published an article on a software bug in a gamma knife device which is used in various cancer treatments. This device is used to focus high levels of gamma radiation at specific points in the body and provide more “surgical” precision to radiation therapy. The bug they found in this case was that the emergency stop button did nothing when pushed. It is supposed to retract the patient from the machine and turn off all radiation emissions. Now the manufacturer has claimed that additional radiation exposure was minimal, one could see how this could create a very dangerous problem for the patients and the staff controlling the device.

This article brought back memories immediately of the Therac-25 incident (and the Wired article points this out as well) in which several patients were given lethal doses of radiation due to software defects and operator misunderstanding. Regrettably, the Therac-25 was developed right in my home town of Ottawa, ON. Not a proud moment in Canadian software history.

These two incidents point out the fact that software can quite easily kill and the need for not only the right standards (e.g. FDA 510(k)), but also the need for proper design and testing. Of course, I’m going to emphasize the need for tools because, unfortunately, exhaustive testing of these devices is extremely expensive in time and money. We still have a long way to go.

Are hardware guys/gals smarter than us?

Do you think we software guys/gals could design something as complex, using our current processes, tools and working methods as the latest Intel Core i7 processor? It has 731 million transistors and is likely one of the most complex man made objects. I seriously doubt it, certainly not in the time frame that Intel’s hardware engineers can.
How many of the IC designers create chip layouts by hand? Hmmm, none. How many milestone builds, alpha and beta runs do they do? Likely very few. How many times do they leave major defects in shipping product or let customers test the end product? Not for long (since bankruptcy would soon follow).

There are many reasons why they can do what they do and software engineers struggle with the complexities of their job. A major reason is the early realization in the field of IC design that tools are absolutely necessary to get the job done. They use very advanced CAD, simulation tools and high level programming languages like VHDL (which is related to Ada). They also depend heavily if not completely on reusable components. No chip manufacturer would survive by reinventing every ALU or cache controller for each new chip. The combination of advanced CAD tools and reusable components is the key productivity enhancer. For creating quality products, IC designers create models of their products and test them thoroughly before committing anything to silicon. Morever, they simply cannot afford to get it wrong. Intel would suffer serious problems if their latest microprocessor turned out to be a dud. It’s a competitive market you can’t afford a false step.

Software engineers rarely use advanced modeling and simulation tools and high-level languages are often scoffed at. Now, someone will likely argue that, “the tools just aren’t good enough! We don’t have that fancy technology that hardware engineers have!” Early in my career, the first large colour screen workstation that I used, in 1989, was an Intergraph Interpro 32c. Guess who these were for? Us software guys? Nope, it was for PCB layout for the hardware engineers.

Software engineers could have tools like the IC designers if there was a fundamental commitment by the profession to use tools, simulation, high level languages and quality driven development processes. I argue that without the demand for the tools that could make the work easier they won’t be built. I’m sure early CAD tools were awful and I’m sure the engineers complained how it was easier to do it by hand. Somebody persevered because they can’t be saying that today. Yet software engineers are just as likely to pick C, Vi and gcc as the only tools they need just as they would have 10 or 20 years ago. They say “those tools can’t do the magic I do with Vi and C!.” If the commitment to changing the way we do software exists, the processes and tools will come.

There is some hope, such as Motorola’s drive to implement their Six Sigma process across hardware and software. I also think that driving this into the software engineering curriculum is necessary and the ACM and IEEE are working on this. See the SE 2004 proposal. However, from my own education, most of the professors knew how to teach programming concepts and languages but didn’t really have much to offer in terms of processes, tools, quality, methodologies, etc. They simply didn’t have the experience or background to back up that learning. Moreover, industry wants engineers who know things like C, assembler, real-time programming not how to do things though development processes, modeling and simulation (unless, of course, they know all the nuts and bolts stuff too!). Our ability to compete and to build the next generation of products depends on an evolution in the way we do things. I’m not seeing it.

The sad state of software development

[I've reposted this from my previous blog - posted May, 2008.]

I’ve just been reading a great paper by Charles Mann called “Why Software is so Bad“. There’s various reasons given such as poor design practices, increasing pressure to deliver on time with more features, lack of customer understanding and confusing and conflicting priorities, requirements, goals, objective, you name it. Two quotes that I really like from this paper are

“Software sucks because users demand it to.”, and,

“We don’t learn from our mistakes…there is no well-defined mechanism for investigating failures and no mechanism for ensuring that people read about them.”

The first quote is from a former Microsoft CTO whose was lamenting that customers both argue against upgrading while simultaneously demanding new features, thus, in the end, causing vendors to cram more features into each release and deliver on time in a competitive market. The second quote is from a Rand corporation researcher looking into the French Arianne 5 rocket launch catastrophe. I highly recommend you read this paper. There are also some great responses to this paper on Edward Tufte’s bulletin board site.

There are many angles to this problem and I think most of us would acknowledge that there is a problem with software development today. Now as a tools product manager, I will put forward that part of the solution is for software engineers to make more use of tools and company management needs to understand the importance of tools and taking time to learn and adapt the tools to their environment and projects. Tools are just that, tools and unfortunately the tools vendors have overblown the claims of their products in the past and present and everyone is skeptical. And rightfully so. However, I think the solution is more along the lines that there needs to be honest commitment to a higher standard and taking to time to do things like adopting tools and training for their use. Now, this doesn’t imply more process or certifications, neither of which seems to have made better products. I mean a commitment to producing a better product which passes the in-house “sniff” test. What does this mean? We can smell that bad smell of a product that’s been rushed out the door. We know, usually openly, within the company that something’s half baked. We also usually know that we’ve based the product on a quick prototype or on a big mess of code from the very distant past. We patch it together, we test it somewhat and we ship it.

It doesn’t have to be this way and the pay off for turning things around can be significant. It also doesn’t mean you scrap everything and start over. It can mean, that you start looking at the tools you already have – are they being used? Or, it could mean, that you start restructuring your software for future churn or you consider making the extra investment in testing and testing tools. Perhaps what we need is a corporate sniff test manager whose job is too blow the whistle on products that are not passing the sniff test. This person doesn’t worry about processes and defect counts, their sole job is to use the product like a customer and say whether the product stinks or not. If it stinks, fix it until is doesn’t.

Product versus requirements management

The product manager role changes from company to company and can evolve while your in the role over time as well. Ideally a product manager should be forward looking, helping guide the company on where the next opportunities are while still making sure the next release is satisfying as many business and customer needs as possible. Where I see a disconnect is in gathering and managing requirements. Surely it is the product manager’s role to feed requirements into the software development engine but does it make sense to make the product manager the requirements manager? The problem with doing so is that the product manager ends up with so much day-to-day details there is no time for the forward looking part of the job.

The reason for this is usually a lack of resources and to some extent a lack of understanding of what product management is. A lack of resources results in a discussion along the lines of “somebody has to do requirements management and well, you’re the product manager, you own the product, you own the requirements.” I’ve seen this many times in the past and the end result of any discussion over who does what is usually “you own the product, therefore…”. Of course, this is all well and good if all parties are happy with the arrangement. In my experience, this is never a stable state for long. After grinding away at the day to day detail of a complex product release, the lack of attention on customers, the market, the trends,  and the general industry “buzz” results in questions as to what exactly is product management doing for the company? I would summarize my recommendations as follows:

• PM  “owns” the product but the level of detail needs to be course-grained. Not every niggling detail needs PM approval.
• Ownership of the product means the PM owns the roadmap and future direction and the responsibility of making sure the product is going in the right place to maximize some positive return (e.g. market share, revenue, customer satisfaction). While I would always want the PM to be in the loop on decision making, other stakeholders can make decisions on a product under development.
• PM is part of the team that provides requirements for a product or a release. Other stakeholders can be, for example, R&D, sales, and technical support. PM may be responsible for gathering these and documenting them but not necessarily the entire life cycle of the requirement.
• There is nothing wrong with another stakeholder owning product requirements, this role is more about making sure requirements are properly addressed in the product and managing the lifecycle of the requirements from creation to validation.
• Conversely, PM can own requirements management as part of the job but something has to give: customer visits, sales support, market analysis, planning, etc. There is a lot of detail work in every product release, you can’t see the forest for the trees.
• PM’s need to be set free from the daily grind of product development details if they are expected to learn of new opportunities, discover customer needs or understand future trends.

What is the experience in your shop? More requirements than product management? Less customer face-to-face and more Excel spreadsheets?

Change is good

I’ve recently been “restructured” and looking for a new opportunity. Although its never easy to be removed from your current money earning position, a change of place and facing a new challenge is good. A couple of things I’ve learned:

1. Read your employment agreement completely, if you can, have a lawyer look at it. Take my word, it may come back to bite you.
2. “Non-compete” clauses are essentially unenforceable but your future employer may be conservative and be wary of any remaining obligations.
3. “Non-compete” clauses can extend beyond your termination, be wary of the time period and how specific the wording is for the clause.
4. Be wary of labor laws of the where you are working. Does this agreement lock you into a particular set of rules? Are you signing away important protections?

I can say from my experience, I unknowingly signed away Canadian federal protections in favor of Ontario’s much more limited protection. Not only that I signed a clause that explicity states the company can terminate without cause! Severance packages for software professionals are usually above the legal minimums but invariably the minimums are set very low. Remember to keep your agreement you signed, file it away and remember where you put it!