by Charles Moore | Jun 7, 2019 | Articles, Artificial Intelligence
Augmented reality virtual elements, virtual reality, artificial intelligence- exactly what are they and how do they interact with one another? Every moment of our waking lives, we use our five senses to learn about our world. In our daily reality, we see people and cars moving on the street, or hear a colleague talking with a client in the next cubicle. We can smell something burning or peculiar fish smells or our morning bacon cooking. Our senses can tell us a lot — but we may still be missing some very important information. If today’s innovators have their way, augmented reality virtual elements will soon fill in those sensory gaps for us.
A Second Intelligence
Your curiosity about this subject is a sign of your own intelligence, but computing machines offer us something different. Artificial intelligence (AI) uses the computing power of machines to perform tasks that are normally associated with intelligent beings. Those tasks include activities related to perception, learning, reasoning, and problem solving. AI can add to our personal experience through something called augmented reality (AR).
We should not confuse the two terms, although they are related. You might compare them to what we know as perception and reason in human beings. We perceive the world through our five senses, but we interpret those perceptions through our reasoning powers. Augmented reality uses devices like smart glasses and handheld devices to provide us with more data and add to our perceptions, but it is artificial intelligence that makes sense of all that information.
What is Augmented Reality virtual elements without AI? It is like eyes without a brain. Tyler Lindell is an AI/ AR/ VR software engineer for Holographic Interfaces, as well as a software engineer at Tesla. In an article called “Augmented Reality Needs AI In Order To Be Effective“, he says that most people don’t realize that “AI and machine learning technologies sit at the heart of AR platforms”.
Another Set of Eyes and Ears
There are some larger questions about the meaning of intelligence and the role of computers that are always good to trigger research and deep conversations. I have written about the history of artificial intelligence and whether machines can actually think. Recently I took another look at J.C.R. Licklider’s vision for man-computer symbiosis. But for those in the business world or in a production environment, you may just want to know what these technologies can do. An article from Lifewire tells us that augmented reality “enriches perception by adding virtual elements to the physical world”.
Just as our eyes and ears need the brain to interpret the sights and sounds that are presented to us, Augmented reality virtual elements depends on AI to provide pertinent information to the user in real time. Imagine taking a walk through the city. You see buildings and landmarks. If you looked through an AR device, it could give you more information, such as the name or address of the building, or some history about the landmark.
Four Categories of Augmented Reality Virtual Elements
An online guide to augmented reality describes four different categories of AR. Marker-based AR (also called Image Recognition) can determine information about an object using something called a QR/2D code. It uses a visual marker. Markerless AR is location-based or position-based. GPS devices might fit into this category. Projection-based AR projects artificial light onto real world surfaces. And superimposition-based AR puts a virtual object into a real space, such as IKEA’s software that lets you see how a couch might look in your living room.
Augmented Reality devices in various stages of development include:
- sensors and cameras
- projectors
- eyeglasses
- heads-up display (HUD)
- contact lenses
- virtual retinal display (VRD)
- handheld
Technology in Transition
The potential of augmented reality virtual elements backed by artificial intelligence is only now being realized in the marketplace. Tech evangelist Robert Scoble and his co-author Shel Israel believe that we are only in the beginning stages of technological development that will have an enormous impact. In their 2016 book The Fourth Transformation: How Augmented Reality & Artificial Intelligence Will Change Everything, they say that we are on the cusp of a new stage. The four “transformations” in their theory can be summarized with these headings:
- Text and MS-DOS
- Graphical user interfaces
- Small devices
- Augmented reality
The technological revolution is already underway. Google’s experiment with smart glasses was an early entry into the consumer AR market. Now augmented reality is being introduced into a broad spectrum of industries, from construction to military. IKEA and other retailers have seen the value of augmenting the views of customers who may potentially place furniture into their homes. Architects and builders are using AR to visualize how new construction might fit into current settings. AR solutions are being developed for technicians in a variety of fields to get analytics in real time. Soldiers with AR visors will be able to get battlefield data as fighting occurs.
The Ironman movies from Marvel Comics give us an illustration of augmented reality. In his high-tech suit, the character Tony Stark sees constantly changing data that he would never have perceived on his own. An artificial intelligence in the suit searches its vast data sources and offers split-second assessments based on immediate events. Like Ironman, AR devices in the coming years will be highly dependent on AI and its resources to aid us in our tasks
Challenges in Augmented Reality Virtual Elements
It takes a while for applied science to catch up with the imaginations of science fiction. There are such limitations as physics that prevent the speedy invention and implementation of the devices on our wish list. The flip mobile phone reminded some people of Captain Kirk’s communicator, but it took a lot of technology to get us there. Ironmen’s augmented reality has a lot more challenges.A short cartoon posted by The Atlantic shows how augmented reality will change tech experiences.
The company Niantec offers a smartphone app that gives you information about the places you visit. “The application was designed to run in the background and just to pop up,” says the narrator.
The next Niantec project was Pokémon GO, an augmented reality game that went viral. The company’s CEO, John Hanke, says that “AR is the spiritual successor to the smartphone that we know and love today.” However clever our ideas, the obstacles can be overwhelming. What happens when Ironman or Captain Kirk lose connectivity? How much bandwidth is required to transmit all that data, and what do we do when transmission channels become congested?
How can AI access the pertinent data quickly enough to be helpful when we need it? And how can we manage all that information?
Conclusion
There are so many potential use cases for augmented reality that go beyond the scope of this article. In the hands of police, the military, or rescue personnel, AR devices could help catch criminals, win battles, or save lives. Devices embedded with image and speech recognition capabilities could become our eyes and ears. Repairmen could use AR to find leaks or diagnose defective equipment. The wonders of augmented reality virtual elements, along with artificial intelligence, will become much more apparent to us in the next few years.
by Charles Moore | May 26, 2019 | Articles, Artificial Intelligence, Wireless Ecosystems | IoT
It’s been more than a decade since the time when the number of internet-connected devices exceeded the number of people on the planet. This milestone signaled the emergence and rise of the Internet of Things (IoT) paradigm, smart objects, which empowered a whole new range of applications that leverage data and services from the billions of connected devices. Nowadays IoT applications are disrupting entire sectors in both consumer and industrial settings, including manufacturing, energy, healthcare, transport, public infrastructures and smart cities.
Evolution of IoT Deployments
During this past decade IoT applications have evolved in terms of size, scale and sophistication. Early IoT deployments involved the deployment of tens or hundreds of sensors, wireless sensor networks and RFID (Radio Frequency Identification) systems in small to medium scale deployments within an organization. Moreover, they were mostly focused on data collection and processing with quite limited intelligence. Typical examples include early building management systems that used sensors to optimize resource usage, as well as traceability applications in RFID-enabled supply chains.
Over the years, these deployments have given their place to scalable and more dynamic IoT systems involving many thousands of IoT devices of different types known as smart objects. One of the main characteristic of state-of-the-art systems is their integration with cloud computing infrastructures, which allows IoT applications to take advantage of the capacity and quality of service of the cloud. Furthermore, state of the art systems tends to be more intelligent, as they can automatically identify and learn the status of their surrounding environment to adapt their behavior accordingly. For example, modern smart building applications are able to automatically learn and anticipate resource usage patterns, which makes them more efficient than conventional building management systems.
Overall, we can distinguish the following two phases of IoT development:
- Phase 1 (2005-2010) – Monolithic IoT systems: This phase entailed the development and deployment of systems with limited scalability, which used some sort of IoT middleware (e.g., TinyOS, MQTT) to coordinate some tens or hundreds of sensors and IoT devices.
- Phase 2 (2011-2016) – Cloud-based IoT systems: This period is characterized by the integration and convergence between IoT and cloud computing, which enabled the delivery of IoT applications based on utility-based models such as Platform-as-a-Service (PaaS) and Software-as-a-Service (SaaS). During this phase major IT vendors such as Amazon, Microsoft and IBM have established their own IoT platforms and ecosystems based on their legacy cloud computing infrastructures. The latter have alleviated the scalability limitations of earlier IoT deployments, which provided opportunities for cost-effective deployments. At the same time the wave of Big Data technologies have opened new horizons in the ability of IoT applications to implement data-driven intelligence functionalities.
AI: The Dawn of Smart Objects using IoT applications
Despite their scalability and intelligence, most IoT deployments tend to be passive with only limited interactions with the physical world. This is a serious set-back to realizing the multi-trillion value potential of IoT in the next decade, as a great deal of IoT’s business value is expected to stem from real-time actuation and control functionalities that will intelligently change the status of the physical world.
In order to enable these functionalities we are recently witnessing the rise and proliferation of IoT applications that take advantage of Artificial Intelligence and Smart Objects. Smart objects are characterized by their ability to execute application logic in a semi-autonomous fashion that is decoupled from the centralized cloud.
In this way, they are able to reason over their surrounding environments and take optimal decisions that are not necessarily subject to central control. Therefore, smart objects can act without the need of being always connected to the cloud. However, they can conveniently connect to the cloud when needed, in order to exchange information with other passive objects, including information about their state / status of the surrounding environment.
Prominent examples of smart objects follow:
- Socially assistive robots, which provide coaching or assistance to special user groups such as elderly with motor problems and children with disabilities.
- Industrial robots, which complete laborious tasks (e.g., picking and packing) in warehouses, manufacturing shop floors and energy plants.
- Smart machines, which predict and anticipate their own failure modes, while at the same time scheduling autonomously relevant maintenance and repair actions (e.g., ordering of spare parts, scheduling technicians visits).
- Connected vehicles, which collect and exchange information about their driving context with other vehicles, pedestrians and the road infrastructure, as a means of optimizing routes and increasing safety.
- Self-driving cars, which will drive autonomously with superior efficiency and safety, without any human intervention.
- Smart pumps, which operate autonomously in order to identify and prevent leakages in the water management infrastructure.
The integration of smart objects within conventional IoT/cloud systems signals a new era for IoT applications, which will be endowed with a host of functionalities that are hardly possible nowadays. AI is one of the main drivers of this new IoT deployment paradigm, as it provides the means for understanding and reasoning over the context of smart objects. While AI functionalities have been around for decades with various forms (e.g., expert systems and fuzzy logic systems), AI systems have not been suitable for supporting smart objects that could act autonomously in open and dynamic environments such as industrial plants and transportation infrastructures.
This is bound to change because of recent advances in AI based on the use of deep learning that employs advanced neural networks and provides human-like reasoning functionalities. During the last couple of years we have witnessed the first tangible demonstrations of such AI capabilities applied in real-life problems. For example, last year, Google’s Alpha AI engine managed to win a Chinese grand-master in the Go game. This signaled a major milestone in AI, as human-like reasoning was used instead of an exhaustive analysis of all possible moves, as was the norm in earlier AI systems in similar settings (e.g., IBM’s Deep Blue computer that beat chess world champion Garry Kasparov back in 1997).
Implications of AI and IoT Convergence for Smart Objects
This convergence of IoT and AI signals a paradigm shift in the way IoT applications are developed, deployed and operated. The main implications of this convergence are:
- Changes in IoT architectures: Smart objects operate autonomously and are not subject to the control of a centralized cloud. This requires revisions to the conventional cloud architectures, which should become able to connect to smart objects in an ad hoc fashion towards exchanging state and knowledge about their status and the status of the physical environment.
- Expanded use of Edge Computing: Edge computing is already deployed as a means of enabling operations very close to the field, such as fast data processing and real-time control. Smart objects are also likely to connect to the very edge of an IoT deployment, which will lead to an expanded use of the edge computing paradigm.
- Killer Applications: AI will enable a whole range of new IoT applications, including some “killer” applications like autonomous driving and predictive maintenance of machines. It will also revolutionize and disrupt existing IoT applications. As a prominent example, the introduction of smart appliances (e.g., washing machines that maintain themselves and order their detergent) in residential environments holds the promise to disrupt the smart home market.
- Security and Privacy Challenges: Smart objects increase the volatility, dynamism and complexity of IoT environments, which will lead to new cyber-security challenges. Furthermore, they will enable new ways for compromising citizens’ privacy. Therefore, new ideas for safeguarding security and privacy in this emerging landscape will be needed.
- New Standards and Regulations: A new regulatory environment will be needed, given that smart objects might be able to change the status of the physical environment leading to potential damage, losses and liabilities that do not exist nowadays. Likewise, new standards in areas such as safety, security and interoperability will be required.
- Market Opportunities: AI and smart objects will offer unprecedented opportunities for new innovative applications and revenue streams. These will not be limited to giant vendors and service providers, but will extend to innovators and SMBs (Small Medium Businesses).
Future Outlook
AI is the cornerstone of next generation IoT applications, which will exhibit autonomous behavior and will be subject to decentralized control. These applications will be driven by advances in deep learning and neural networks, which will endow IoT systems with capabilities far beyond conventional data mining and IoT analytics. These trends will be propelled by several other technological advances, including Cyber-Physical Systems (CPS) and blockchain technologies. CPS systems represent a major class of smart objects, which will be increasingly used in industrial environments.
They are the foundation of the fourth industrial revolution through bridging physical processes with digital systems that control and manage industrial processes. Currently CPS systems feature limited intelligence, which is to be enhanced based on the advent and evolution of deep learning. On the other hand, blockchain technology (inspired by the popular Bitcoin cryptocurrency) can provide the means for managing interactions between smart objects, IoT platforms and other IT systems at scale. Blockchains can enable the establishment, auditing and execution of smart contracts between objects and IoT platforms, as a means of controlling the semi-autonomous behavior of the smart object.
This will be a preferred approach to managing smart objects, given that the latter belong to different administrative entities and should be able to interact directly in a scalable fashion, without a need to authenticating themselves against a trusted entity such as a centralized cloud platform.
In terms of possible applications the sky is the limit. AI will enable innovative IoT applications that will boost automation and productivity, while eliminating error prone processes. Are you getting ready for the era of AI in IoT?
