Ready, Steady, HTML5, Go! (or How I Used HTML5 to Build My TV and Survived)

Where do I start from? That is the question that user interface developers should be asking themselves when starting a new project.

JavaScript is a good place to start. I have previously claimed that JavaScript will be used to develop the next generation of consumer electronics applications. But JavaScript is just a programming language. It is the rawest of raw ingredients, the lowest common denominator required to put together a complete solution. In building a complex software system it would be better to start with more than just the most basic of building blocks.

OEMs who have product in the market probably have existing software that has been developed internally over time. However the game is changing and the technology needs to change to keep pace. New services must be enabled, OTT is crucial, social network and recommendation engine integration is expected, live interaction with online services is normal.

Most importantly, a dynamic, dazzling and compelling user experience is the key to winning customers

The old, internally created, software components won’t cut it anymore, greater flexibility and better features are needed.

To address this, the chipset providers have begun offering Webkit and HTML5 based development as part of their SDKs in the hope of providing familiar tools to developers by way of a JavaScript / HTML API. But again, this is a bare-bones starting point.

Trying to build a sophisticated User Experience starting with raw HTML5 is like trying to build a skyscraper with gravel, sand and a shovel

Some OEMs and semiconductors are turning to Open Source Web Components to give them this advanced starting point. While this might seem like a good idea I have listed here some reasons why this is in fact not a good idea, unless you are set up in a particular kind of way, and why it is usually more risky and expensive than sourcing something purpose built.

Not Fit for Purpose #1 (Resources)

Consumer Electronics software is a form of Embedded Software as discussed previously. A TV or Set Top Box is not as powerful as your desktop computer. Software that runs well on the desktop with a quad core Intel i5 clocked at 3.5Ghz and 6Gb memory will not automatically work well on a single core MIPS CPU clocked at 400Mhz with 256Mb memory. How could it?

Assumptions built into the design of the Web Components about resources, such as available memory (infinite on the desktop and scarce on the STB) and CPU speed, are invalid. For example, if you need to manage a screen that is 1080p resolution you need graphical assets that fill that screen. One screen worth would consume close to 8Mb of memory (1080p resolution at 32bpp). If you want a background image behind all that then you can allocate another 8Mb, and if you want multiple screens so that you can create cool transition effects when moving from one screen to the next you need multiples of that 16Mb.

Of course there are corners that may be cut to reduce memory consumption but that’s exactly the point

Corners need to be cut to reduce resource utilisation on embedded systems while on the desktop they do not. Web Components designed for desktop applications will require a lot of work to optimize for Consumer Electronics, if they can be made to work at all.

Optimizing for embedded is all about cutting those corners

Not Fit for Purpose #2 (Graphical Input and Output)

A TV and a desktop computer have completely different graphical input and output models. The TV is designed to be driven by a 5-way button remote control while the desktop computer is driven by a mouse. Consider “hover”, used on web pages to get drop down lists or pop up menus. There is no “hover” with a remote control. Software components that are graphical and meant for human interaction, such as menus, lists and text entry, etc. will require considerable work, and possibly complete redesign, to operate correctly for TV. Or to put it another way, to reach a simple drop down menu item designed for a mouse, using a remote control, might require the following keypress sequence: up-up-up-right-right-enter-down-down-down-right-down-down-enter. Not exactly convenient.

UI components designed to work with the mouse will not work with the remote control

A Disintegrated System?

In addition to getting Web Components to work well on an embedded target they need to work well with each other. A cohesive software architecture needs to be in place so that the entire software system may be programmed easily and work with minimal overhead. Software components from different sources are pretty much guaranteed to work differently from one another and a lot of development effort and CPU cycles will be directed to converting between them and ensuring the system as a whole works as it should.

For example if one component provides an interesting menu system but requires a particular type of bi-directional list to work, but the content backend provides the data as a hash table, then precious CPU cycles will be spent transforming data to and from those data types. Or say the Cool-Transition-Effect Web Component manipulates image data in a destructive way, so a copy of the image data needs to be taken before passing it off.

More CPU cycles, more memory, more overhead!

If you are going to cobble bits of software together you will be left with a mish-mash of glue code and a mess.

Why Do I Need a Consistent SDK API Anyway? Semiconductors Do

As an extension to the previous point, consider that semiconductor chipset manufacturers offer an SDK to their developer customers. Their SDK API is the window to their product and hence a critical technical and marketing tool.

Semiconductors are judged by the consistency and usefulness of their SDK API

FancyPants is also represented by an API. As flexible and powerful as the underlying FancyPants engine is, the value of the technology is first and foremost embodied in the API.

When bringing together components from disparate sources the API of each component will usually be different, hence work will have to be done to bring these APIs into a consistent form. This may, or may not, be possible depending on the components themselves.

Open Source Economics

If a company is geared up to working with Open Source then it may make sense to use Open Source Web Components. But deciding on this path needs to make sense to the business. There needs to be a strategy in place for working with the Open Source community and it usually helps if there is a strong involvement with the community from which the Web Components are sourced.

While the benefits of using Open Source are stated as having access to the solution of a complex problem, or a mature component of code, there are still many costs involved.

Open Source is not “Magic Code”, it must be reviewed, optimized, fixed, integrated, tested, and more

The tasks involved when working with Open Source should mirror those tasks already in the software development process minus the actual development itself not counting the Q/A cycle [1]. Your overheads come from having to understand code written by someone outside the team and for a different purpose, optimizing that code, refactoring it, integrating it, making it consistent with the rest of the code, etc.

And what about this “mature” benefit? If we are talking about the Linux Kernel, then yes, this is mature. If we are talking about a Web Component making use of the latest Webkit API then maybe not. If this is going into a TV, then ask how many consumer electronics products has this Web Component actually been deployed in? Do you want to be the first?

The point is that people in the team will need to take ownership of the code, much as they do for code developed internally, and be responsible for the Open Source code and all that comes with it.

FancyPants: Fit For Purpose

At Fluffy Spider Technologies, we have recently licensed our Graphical User Interface framework, FancyPants, to Sigma Designs. OEMs using any Sigma chipsets have a great head start as the Fluffy Spider FancyPants framework seamlessly integrates with their chipset capabilities providing an easy to use, high level API.

In general, by using a higher level framework you get the benefit of more complete building blocks which translates to shorter time to market and greater confidence in the software. By using a higher level framework that is targeted at solving a specific problem you get the added benefit of that framework being fit for purpose.

Meet Me at IBC

If you would like to meet at IBC, Amsterdam, we will be there exhibiting with semiconductor partners, so please drop me a line:


[1] At FST we allocate around 70% of development time to Q/A

Jul, 24, 2012