I have a simple question: what if we built audiovisual systems as browsers?
I don’t mean connecting a laptop or PC to a large screen. I’m not referring to supporting BYOD. I mean, very literally, what if the AV system WAS the browser? Laptops, desktops, tablets, phones - they ARE the browser in the sense that the browser has control over the physical hardware. Of course, performance and capability vary across hardware, but when the browser has focus, the hardware becomes an extension of the content within it. Browser based applications behave like native device applications.
That’s a very powerful concept: hardware as an extension of the content. Browser-based gaming, WebRTC, gestural controls demonstrate some of the power, and that’s all confined to a device you can hold in your hands or sits on your desk. Imagine what might be possible at human scale, at architectural scale. Content could adapt appropriately and automatically from tablet, to conference room (certainly achievable today), but also into auditoriums, or across architectural media elements. It could behave differently across different canvases, or with different audience characteristics. One piece of content, one presentation, could be utilized across all of those venues.
Stories power society. Stories drive the world forward. They compel the stagnant to action. They calm the anxious. They incite riots. They make heroes, martyrs, villains. They facilitate loyalty. They instill confidence and trust. They drive commitment. They drive desire and reveal need (or create the perception of need). They cause spending and saving. Ultimately, the AV industry facilitates telling an infinite spectrum of stories. Boardroom stories differ from retail stores. Command and control stories differ from three-hundred seat auditorium stories. We match equipment and software to design a wide range of story-telling systems.
Isn’t funny how many of those systems end-up displaying PowerPoint? PowerPoint is ubiquitous. It’s low friction. It scales in size to fit any screen. It’s immensely valuable and the butt of jokes. And it illustrates two critical truths about AV solutions. First, content is always the singular reason for building systems. That’s made clear when you discover the user has resorted to using PowerPoint to drive digital signage or to deliver content to million-dollar LED walls. “This just worked,” they tell you. Secondly, even well executed systems can look like a failure when their capabilities lie unused. This isn’t intended to knock PowerPoint. Its adaptability is a testament to its value. And it’s not to say those underutilized AV systems are failures. What it reveals is a giant capabilities gap. We can build fantastic environments for storytelling, but it can be difficult to fully utilize them. It’s costly, it’s time-consuming, and it relies on limited resources.
Obviously, we use the browser in AV systems now. If you build a presentation as a website and it will display on any browser. Almost no one works this way, though. Several of the PowerPoint alternatives are simply web presentation tools. Google Slides is winning users over because of the ease of collaboration. There isn’t enough motivation to push many users or user organizations out of their comfort zones. There are three major reasons why the browser-based presentation hasn’t become more integral to AV systems design.
First, the appetite for funding rich, highly engaging presentations has historically been concentrated on large spaces like auditoriums or high touch environments like executive briefing centers. These are characterized by large display systems, often in non-standard aspect ratios or resolutions, and by workflows requiring specialized skill sets. The video processors required to drive large display systems fall into two categories. Hardware-based processors are windowing and compositing beasts. They are fast, high-bandwidth, highly resilient, and expensive. They don’t render graphics like a PC, instead they manipulate the pixels fed through them. Software based video processing runs on standard computer hardware and use high-performance PC graphics cards (think gaming PCs). They excel at rendering content, but they too can process pixels fed through them and composite that video into the content they render. There is more latency than with the hardware processor, but they have a broader feature set. In fact, they share a lot of capabilities with the ordinary web browser, including the ability to render web pages.
Workflow requirements in these environments vary widely. A space be considered self-service, where the presenter or collaborators operate the system themselves. Or a single AV tech may provide support to change window layouts, mute microphones, and advance PowerPoint slides. In complex show flows an expert crew may be required to drive involving multiple cameras, multiple video feeds, sounds, lighting, play out, graphics, and stage direction. Complex AV systems often partially consolidate user or technician controls into a single control surface through an AV control system. On the theatrical or broadcast periphery of AV, more automation can be achieved through show control systems. Consider how show control system are intrinsically tied to the specific nature of the content, often synchronized to time code or cues in a way that typical AV control is not.
Both the video processing requirement and the complexity of live production limit the effectiveness of the browser alone. Designing live production systems are a discipline in themselves, with a broad array of solutions on the market. When setting out to build this kind of venue, (almost) no AV professional says, “let’s do this all in the browser,” although there is an explosion of browser-based workflows emerging where the focus is on control interface in the browser. Such workflows still need to leverage a non-browser rendering tool.
The second reason the browser hasn’t become more integral in AV spaces is the relative immaturity of the ecosystem. While much of the browser feature set is mature and proven, video and audio capabilities continue to evolve. Many features remain experimental or are not fully supported across all platforms (both browser and operating system). The graphics power of the browser is still less than an application specific native application, though recent advancements in capability are very impressive. In short, it’s only in the last few years that the browser has reached a point to where we can even consider it in this capacity. This is the result of the consumer electronics arms race. Video streaming, video chat, gaming, and AR/VR applications are all able to be delivered via the browser, and these advancements will continue to expand the browser impact in AV.
The third reason the browser hasn’t dominated the AV world is mostly a follow-on effect from the first two problems. There simply aren’t any tools for the average presenter which offer a better value proposition over the current ones. Non-PowerPoint users either simply don’t use a lot of presentation materials, they develop them in a similar application like Keynote, or they use a more specialized application like InDesign. While there is growth around Google Slides, it doesn’t offer new capabilities on the presentation side. High-performance users utilize the previously mentioned visualization platforms where advanced control over the AV systems is possible, though the learning curve can be steep, and the available resource pool is limited.
Yes, the browser is used, but functionally it has been no different than a standalone application. As they say on Wall Street, however, historical performance does not guarantee future performance.
Responsive Web Design
Backing up a bit, what enables the adaptability of web content across PC and mobile hardware? Responsive Web Design is a design pattern that leverages standardized browser APIs to create content portability. It is a critical factor in the modern web experience. RWD treats content fluidly, adjusting layout and content scaling according to the capabilities of the device and browser on which it is rendered. Frameworks and platforms, often in combination, both simplify and empower the delivery of that content. There are four major components to this capability:
1. The browser controls the physical hardware through the operating system
2. The browser provides capabilities to the content through (mostly) standardized API's
3. The content is delivered modularly as HTML5, CSS3, Javascript, and related assets and rendered as a final composite by the browser
4. A myriad of browser-side and server-side frameworks and platforms eliminate redundant work and allow both coders and non-coders to focus on the content, driving both efficiencies and innovation
Template based web site hosting platforms such as Squarespace demonstrate the power of this architecture. Users with little knowledge of web development can easily create a modern, elegant website that displays properly across different devices and integrates with other services. Web design tools like this are sufficient for most small business and many large ones. It effectively democratizes the website as a story telling tool, but this democratization ends at the desktop.
Of course, we can plug the laptop into the AV system and display it on any screen of any size. We can do that wirelessly, and we can do it with mobile devices. In fact, these capabilities have been an area of considerable advancement and revenue opportunity with the AV industry over the last few years. But in each case, all we can really do is replicate the browser’s host device across a larger system. Google Slides don’t push presentation capabilities because the browser has no real knowledge of the AV system. But what if it did? Could we extend the RWD concept to create Responsive Audiovisual Design?
Responsive Audiovisual Design
Responsive Audiovisual Design (RAD) would mean replicating the 4 major components that contribute to the RWD mentioned earlier across the AV ecosystem. It’s a proposition for which the AV industry is well suited. Imagine turning a Chromebook inside out, and that you can shrink down and fit inside of it. It looks like the archetypal AV system. There is a display, a camera, a microphone, speakers, touch interfaces, some processing, some software, and a connection to a network. We may not design and build Chromebooks within the AV industry, but we definitely deliver systems that look like this inside out version.
1. We provide operating systems for our systems in the form of AV control systems. These are morphing into modern web architectures with rich native APIs
2. Chromium is an open source version of Chrome that can be extended and integrated into other applications, so we have a ready-made browser available to us. But we don’t necessarily need a custom browser, as the modern browsers all support the HTML5/CSS3 standards allowing enormous expansion and flexibility. We can easily develop a script that extends the native browser APIs that provide access to PC hardware to also provide access to our AV system hardware (via the control system)
3. The modularity of HTML5/CSS3/Javascript allows access to additional resources and to recognize the larger canvases found in AV systems
4. Those frameworks and platforms remain valuable and can be leveraged across a whole new tier of endpoints. Even more promising is our ability to develop new platforms to fully leverage the immersive nature of human and architectural scale systems
Putting the browser at the center of the AV system makes it the final rendering point before the display system. The content is delivered to the browser modularly along with rules on how to display it. Scripting tells the browser how to manage the content, deal with interactions, and how to use the resources in the larger AV system. The browser doesn’t just scale or stretch content, it reorganizes the content appropriately, leverages extra audio channels or manages changes according to sequenced rules or interactions. It marries the content to the system, giving the user, the story-teller, the full power of this experience delivery machine. Powerful consequences emerge.
1. Web development resources are much greater resource pool and available to more stakeholders as compared to any proprietary desktop application
2. Packaging the content within the HTML ecosystem guarantees portability of the content across touchpoints
3. The efficiencies of leveraging the seemingly endless stream of new web components, frameworks, and platforms coupled with the universal portability of content in HTML creates a positive feedback loop allowing more innovation and capability in the AV systems
4. All of these result in better empowered story-tellers, whether it is a talk in front of hundreds, or a shared collaboration session in a huddle space
5. Modern web architectures consider security as a top priority, and the modularity of the content allows more granular control of information. For enterprise and small business, data governance is vastly improved over proprietary, native application presentation systems
How do we build RAD systems then? As noted, all of this is well within the capabilities of the AV industry, and well aligned with the path ahead of us. I would go so far as to argue that this is inevitable, as the browsers further consolidate their role as the universal application shell and AV system components convert to software or simply disappear. The dependencies that must be resolved and a path toward fully responsive AV systems will be explored in another article, “The Foundations of Responsive Audiovisual Design.”