SmartDeviceLink Application to Intelligent Climate Control

Oleg Gusikhin, Omar Makke, Jeffrey Yeung and Perry MacNeille

Research and Advanced Engineering, Ford Motor Company, 20300 Rotunda Drive, 48121, Dearborn, Michigan, U.S.A.

Keywords: Air Quality, Climate Control, Machine Learning, Intelligent Control, Connected Vehicles.

Abstract: SmartDeviceLink (SDL) is an open-source software development kit (SDK) that enables a smart-device to

connect to the vehicle, providing functions for safe and easy access to the vehicle human-machine interface

(HMI) and the ability to programmatically control vehicle functions. This paper discusses a framework for

developing intelligent control applications that implement personalized and context-aware features for

automotive climate control systems. There is also a discussion of integration of wearables, internet-of-things

(IoT) sensors, cloud and mobile machine learning.

1 INTRODUCTION

Connected cars and Internet of Things (IoT)

technologies are at the frontier of the automotive

industry innovation (Lu, 2014; Swan, 2015). In recent

years, significant progress has been made in the area

of brought-in connectivity. The increasing ubiquity of

smartphones created a need to provide a safe and easy

access to smartphone apps while driving. Ford Motor

Company’s SYNC

Applink was the first wide scale

implementation of such technology followed by the

rest of the industry generating a multitude of different

solutions. This situation presented a significant

challenge for the app development community as it

requires adaptation of a given app to each

automaker’s interface.

To address this challenge, Ford contributed

Applink to the open source SmartDeviceLink project

to promote the development of an industry wide

standard. MirrorLink is another phone linking open

standard that has been developed in parallel with

Applink. It projects the app’s display of a MirrorLink

enabled phone to MirrorLink enabled vehicle head

unit. This standard, however; is not universally

adopted by OEMs and phone providers. For instance,

iOS does not support MirrorLink, leaving out the

iPhone users who are a substantial portion of the

addressable market.

Apple and Google also introduced their phone

projection technologies, CarPlay and Android Auto

respectively, that provide in-car access to selected

approved apps through a familiar Android and iPhone

display (Shelly, 2015). In contrast, SDL allows

seamless integration of mobile apps into the vehicle

head unit while maintaining the look and feel of the

given OEM human machine interface (HMI) design.

In addition, SDL implements the capability to

programmatically manage the vehicle sub-system

settings, including radio, climate control, navigation

and other user-configurable infotainment and

convenience features. Consequently, it provides a

powerful platform for cyber-physical systems that

integrate wearables, IoT sensors and cloud data into

the intelligent vehicle control (Smirnov, 2016). The

paper discusses the opportunities for the SDL

application with automotive climate control.

Climate comfort in the vehicle cabin is achieved

through a system of integrated heating, ventilation

and air conditioning (HVAC), either controlled

manually or automatically (Daly, 2011). The basic

HVAC system maintains a manually set temperature

and speed of air flowing from the ventilation system.

On the other hand, an automatic climate control keeps

a set temperature within the space of the cabin by

regulating a blower speed and direction based on

temperature, humidity and sun-load sensors. The state

of the art automotive climate control features multi-

zone automatic climate control. Many high end

vehicles include separate climate control for driver,

front passenger and rear passenger zones. A number

of luxury vehicles have a four-zone automatic climate

control system with infrared sensors that monitor

occupants’ surface temperature. Further advancement

of climate control systems includes the development

of air quality control. Many automakers have now

begun to install air quality sensors in the vehicle that

234

Gusikhin, O., Makke, O., Yeung, J. and MacNeille, P.

SmartDeviceLink Application to Intelligent Climate Control.

DOI: 10.5220/0005997002340240

In Proceedings of the 13th International Conference on Informatics in Control, Automation and Robotics (ICINCO 2016) - Volume 1, pages 234-240

ISBN: 978-989-758-198-4

measure different pollutants in the outside air, such as

2.5

(particulate matter less than 2.5 micron in

diameter), carbon monoxide (CO), and hydrocarbons

(HC). When a high level of pollution is detected,

climate control switches to recirculation mode to

prevent polluted air from entering the cabin.

Factory-installed climate control equipment is an

automotive grade system that must meet a wide range

of requirements for safety, robustness, and

manufacturability. Automotive grade components

withstand significant physical forces, function under

a wide range of ambient conditions, and have a

lifespan substantially longer than typical consumer

devices. The ability to effectively integrate consumer

devices into vehicle control systems not only reduces

the cost of the feature, but also ensures that customers

can leverage the latest technological innovations

available in the market. For example, biometrics from

wearable devices could provide more accurate

prediction of the perceived comfort levels of the

passengers.

The IoT revolution is a game changer for the

control approach. Home climate control has seen such

changes with products like the Nest Learning

Thermostat that implement model predictive control

(MPC) algorithms. IoT technology provides

significant opportunities and challenges for

automotive control applications. Potentially

significantly advances in personalization and

intelligence are achieved more rapidly and at lower

cost than development of the automotive grade

technology. It supports the trend in mobility for on-

demand transportation and a shared car economy, as

personalization is not tied to the vehicle but stays with

a user’s smartphone and can be dynamically

integrated into vehicle control. The ability to collect

and analyse data in the cloud can provide new data

from crowdsourcing of environment information,

remote prognostics and diagnostics and provide new

insights into consumers’ behaviour and climate

control usage. However, it raises a number of

concerns regarding security, safety and robustness of

the applications that rely on external information.

SDL allows integration of IoT technology in a

safe and secure way. It also ensures that the OEM is in

control of what application is used within the system.

This paper examines the way wearable devices,

IoT sensors, personal smart mobile devices and cloud

information can be integrated into automotive climate

control using SDL. These examples show how the

IoT approach can implement intelligent control for

basic HVAC units or augment the existing automatic

climate control system with additional sensory input

and personalization. In the next section, we describe

the SDL APIs relevant to build climate control apps.

Section 3 provides an example of the intelligent

climate control integrating wearables and machine

learning. Section 4 describes air quality control

leveraging brought-in sensors and cloud data. Section

5 provides a summary and discussion of benefits.

2 SmartDeviceLink FOR VEHICLE

CLIMATE CONTROL

SmartDeviceLink is an open source project under

GENIVI Alliance. It comprises head unit software

and mobile SDKs for Android and iOS, as well as

cloud configuration. It supports several transport

protocols: Bluetooth, WiFi and USB.

SmartDeviceLink supports both media and non-

media apps. Media apps are dedicated to audio

streaming and provide alternative user interface (UI)

to the native media UI, which usually include

FM/AM/XM, and CD. Non-media apps normally

read vehicle data and provide added functionality to

the driver. The head unit defines four states called

HMI_LEVEL for each app: FOREGROUND,

LIMITED, BACKGROUND, and NONE.

When the app is selected from the head unit, it

opens a UI and is put in FOREGROUND state, which

gives the app all its permissions. Once opened, the

driver may switch to a different screen on the head

unit, such as the navigation screen, which puts the app

in LIMITED, and thus limiting some of the app’s

permissions.

An app in NONE state is idle and is only allowed

to be discovered and started. The app is put in

BACKGROUND state if its functionality interferes

with a higher priority function, such as an incoming

phone call during audio streaming, and can be treated

as a temporary NONE state for many cases. Mobile

applications can communicate with SDL core once

they implement the SDL software development kit

(SDK), which is available for Android and iOS

platforms. The SDK makes the app discoverable by

the vehicle’s head unit. It exposes a set of remote

procedure calls (RPCs) through a defined set of

application programming interface (API).

In brief, the app instantiates an instance of SDL

proxy class which handles the communication

between the app and the vehicle. The RPCs are

methods of the proxy class. Moreover, the proxy class

intercepts the vehicle’s notifications and makes them

available for the mobile app.

The proxy class also allows the app to query the

head unit for capabilities, since the app can be

SmartDeviceLink Application to Intelligent Climate Control

235

Table 1: SDL API’s for reading and writing module parameters.

API Parameters Description

getInteriorVehicleCapabilities()

Zone

Returns supported modules the vehicle is equipped

with (Radio, Climate Unit…).

getInteriorVehicleData()

Zone, Module

Reads module data. Modules are obtained from

getInteriorVehicleCapabilities.

setInteriorVehicleData()

Zone, Module,

Data

Control API. Sets the data of the Module for the

specified zone

running on different vehicles designed by different

OEMs with different capabilities. Although the RPC

implementation is not directly exposed to the app

developer, it is worth noting that the RPC protocols

are implemented as JSON strings.

A remote control extension for SDL is created and

is available in the public repository. The extension

consists of additions to SDL core inside the vehicle

and to the mobile SDK for Android and iOS. Three

major RPC’s responsible for remote control, and their

corresponding APIs are shown in Table 1. These APIs

provide enough abstraction for mobile apps to control

vehicle modules with different capabilities inside

vehicles by different OEMs.

SDL opens new opportunities to bring IoT to the

vehicle using remote control APIs. Figure 1 shows a

general configuration of how this can be achieved.

Two consumer grade sensors are brought by the

driver: the mobile device hosting the mobile

application, and the optional sensors. The sensors

may be embedded directly on the device hosting the

mobile application, such as an accelerometer or skin

temperature sensor on a wearable, or it may be a

completely separate device, such as a PM

2.5

sensor,

which can broadcast its values over Bluetooth.

Figure 1: SDL-based sensor integration.

The remote control SDK allows reading and

controlling the parameters shown in Table 2 (except

for ambient temperature which is read only) for the

climate control unit. Other modules, such as radio,

also have defined set of parameters which can be read

and controlled. For each and every parameter,

permissions to read and write to the parameters are

controlled by the OEM through a cloud server.

Table 2: SDL parameters for climate control.

Parameter Description

acEnable

Toggles AC ON/OFF

desiredTemp

User input (the set

temperature in the

head unit).

fanSpeed

Blower speed in %

currentTemp

Outside Ambient

Temperature

temperatureUnit

Unit of the

temperatures

circulateAirEnable

Air recirculation

ON/OFF

autoModeEnable

Auto mode value

ON/OFF

defrostZone

Front, Rear defrost

ON/OFF

dualModeEnable

Dual mode is

ON/OFF for units

supporting zone

control

In order to provide control over which apps can be

discovered by SDL and what the app can execute, the

OEM implements a policy table in the cloud. Each

approved app is assigned an application ID (App ID)

by the OEM. Each App ID is associated with an

explicit set of RPCs which the app can execute in each

of the four HMI_LEVEL states of the app. If the app

attempts to execute an RPC that is in incorrect state,

it will be denied. The policy table contains this

information for every approved app in the cloud.

When an SDL app connects to a new vehicle, it will

send its App ID, and the vehicle will search its local

policy table for it. If the App ID is not found, the

vehicle requests the mobile app (the mobile software

development kit specifically) to obtain encrypted

policy information specifically for the App ID. This

request is also initiated regularly to update the local

policy table for all discovered apps which give the

OEM ultimate control of permissions.

ICINCO 2016 - 13th International Conference on Informatics in Control, Automation and Robotics

236

3 PERSONALIZED CLIMATE

CONTROL

The goal of intelligent climate control is to maintain

the comfort level of the user with minimal user

interactions. Most existing automatic climate control

systems maintain a preset temperature by blowing hot

or cold air until the temperature reaches the preset

level. This mode is designed to work for the vast

majority of the population. However, user climate

comfort preferences vary based on physiological and

psychological factors and may change during driving

(Rosenfeld, 2015). In Kuang, 1995, the authors have

demonstrated the perception of comfortable

temperature correlates more with skin temperature

rather than with the ambient temperature of the cabin.

The paper describes experimental control system that

uses an IR sensor to obtain skin temperatures.

Nowadays, we can leverage many wearable devices,

such as the Seraphim Sense Angel Sensor (Seraphim,

2016), that provide a real-time reading of skin

temperature among other biometrics. SDL-enabled

application integrating wearable’s data can be written

to personalize climate settings using cloud services

and machine learning algorithms.

Figure 2: Personalized climate control.

Figure 2 shows the main components of such a

system. It has been demonstrated that neural networks

have been an efficient and effective approach to

implement complex non-linear models for

personalized climate control (Thomas, 2007; Kajino,

2000). The modern smartphone’s hardware provides

sufficiently powerful computational platform to

implement machine learning, and neural networks in

particular. Lane, 2015, discusses a feasibility of

implementation of low power consumption cloud-

free deep neural networks for smartphones for audio

processing.

Recently, Google open-sourced its TensorFlow

™

engine, which can be used to implement deep learning

and neural networks on many platforms, including

locally on smartphones.

The proposed system implementation diagram is

shown in Figure 3. The input includes vehicle climate

control parameters shown in Table 2 which are

obtained via SDL, combined with the skin

temperature from the wearable device, and the output

is the target temperature, which can be set via SDL.

With a proper learning model, the cabin settings could

be adjusted seamlessly to the user’s desires without

user input.

4 CABIN AIR QUALITY

CONTROL

Air quality is increasingly becoming a concern

particularly as urban areas continue to grow. It is

typically described by an Air Quality Index (AQI),

which is a measure of the concentration of various

gases and particulates in the air over a specific time

span and an indicator of the health impact of the air.

Cabin air quality can be improved through proper

management of the climate control system (Müller,

2011). For instance, if the external air quality is

poor, the vehicle should recirculate internal air.

However, recirculating air too long can cause fogging

of windows and drowsiness.

Conversely, if the internal air quality is poor, the

cabin air should be purged with external air by turning

recirculation off and increasing the air blower speed.

Since this air is processed through the vehicle’s air

filter, this will improve the air in the cabin even if the

external air is not clean.

With a growing interest in cabin air quality

management, automakers are actively seeking

implementation of internal and external air quality

sensors with the climate control system. While there is

substantial progress in the development of air quality

sensors for the consumer market, their availability for

automotive applications is currently limited. In lieu of

an embedded system, SDL offers an efficient and

effective approach leveraging brought-in consumer

sensors and cloud-based AQI data.

In the prototype, we used a sensor from

ChemiSense (ChemiSense, 2016) that detects several

analytes using proprietary sensor arrays and machine

learning, including: NO

, NH

, CO, CHOH

(Formaldehyde), Humidity, VOCs (volatile organic

SmartDeviceLink Application to Intelligent Climate Control

237

Figure 3: Architecture of intelligent climate control.

compounds); temperature with a thermistor, and

2.5

with an IR smoke detector. An AQI value is

determined by calculating the indices for CO and

PM2.5. The higher value is taken as overall internal

cabin AQI.

The brought-in air quality sensor is not

necessarily tied to any particular provider, except that

the sensor must communicate with a mobile device or

the HMI with a low-energy radio such as Bluetooth

LE to reduce power draw so the device can be self-

standing. In fact, this platform can work with any

many air quality sensors, provided it meets sufficient

standards for reaction time, for specificity and

accuracy. The interface with the AQI value from the

in-vehicle sensor (and the sensor itself) built with

Applink is shown in Figure 4.

Figure 4: Air Quality Index App and Sensor.

Estimation of the air quality outside the vehicle

was done using the API provided by AirNow

(AirNow, 2016), a website operated by the U.S. EPA

to provide real-time and forecasted data on the air

quality in the U.S. This API provides the calculated

AQI for O

, PM

2.5

, and PM

. The AQI values are

also assigned a category, ranging from “Good” to

“Hazardous”.

For the purposes of this application, we are

focused on PM

2.5

, since this is of the most interest in

China. Thus, wherever AQI is concerned for the

external (outside) readings, it is based solely on

2.5

. AirNow uses weather stations scattered

throughout the U.S. to provide their measurements.

The API determines the closest station to the latitude

and longitude coordinates requested, which we

determine from the GPS of our device. For

deployment in China, a similar governmental service

can be used instead. Obtaining data from AirNow or

similar government data services in other regions is

easy and reliable. However, it is not sufficiently

granular for effective air quality management.

Figure 5 provides the results of the field test

conducted in a Bay Area. During the field test we

identified 3 areas with elevated level of pollution

corresponding to poor air in an AQI chart. At

Dumbarton Bridge (1), a pungent smell similar to

rotting fish and sulfur was noticed. Near the San

Francisco International Airport (2), the air quality

worsened. In downtown San Francisco (3),

measurements were taken on a street by street basis

to formulate a true air quality map during a high-

traffic period. Note that the entire region listed as

good air quality on AirNow even though large areas

of noxious odors were observed.

One way to capture the fixed local areas of

elevated pollution is to remember GPS coordinates of

those areas during repeated drives. These coordinates

can be used to switch to recirculation. Furthermore,

combining government air quality data from the fixed

sources and crowdsourcing from the vehicles together

ICINCO 2016 - 13th International Conference on Informatics in Control, Automation and Robotics

238

with weather, traffic, maps, machine learning and

comprehensive analytics can enable development of

granular and accurate air quality maps that can be

used for air quality management. Even for vehicles

with external air quality sensors, this map can provide

significant benefits: it can provide advance warning

to close windows, start recirculation in advance vs

reacting to AQI reading that due to delay may cause

some polluted air entering the cabin, and provide an

advice on the least polluted routes.

5 CONCLUSIONS

The paper discusses the opportunities to augment

automotive climate control with a SmartDeviceLink

enabled smartphone app.

Figure 5: SF Bay Area Air Quality Test.

Such an approach leverages cloud data, IoT

sensors, and wearable devices with opportunities for

intelligent algorithms personalized to individual

drivers. This paper reviews two potential

applications: integration of wearable devices to

enable personalization of climate control and an air

quality management system based on brought-in air

quality sensors.

Such an approach leverages cloud data, IoT

sensors, and wearable devices with opportunities for

intelligent algorithms personalized to individual

drivers. This paper reviews two potential

applications: integration of wearable devices to

enable personalization of climate control and an air

quality management system based on brought-in air

quality sensors.

Integration of wearables that measure biometric

data adds a new dimension to climate control by

implementing a passive feedback loop requiring

minimal user input. Many wearable devices today

monitor skin temperature in real-time. With this data,

along with information about the current climate

control settings and external weather data, the system

can learn the user's preferences. For example, when

the user increases the cabin temperature, it may relate

to their skin temperature being cold. After this occurs

several times, the system can learn this preference and

automatically increase the cabin temperature when

the wearable detects a cold skin temperature.

Combining this with machine learning, we can create

an intelligent, personalized climate control system.

Correlation of other parameters monitored by

wearables, including metabolic rate, heart rate, stress

levels, breathing, or even blood oxygen levels can

give insight into how climate control affects patients

with chronic illnesses such as asthma.

Cabin air quality management is gaining attention

especially in regions with high pollution levels, such

as China. Despite the innovation of consumer air

quality sensors in recent years, the availability of

automotive grade sensors remains limited. The

proposed solution augments the climate control

system with consumer-grade sensors, cloud-based

information, and applications that manage air

recirculation and refresh.

SDL’s ability to dynamically integrate consumer

devices with vehicle climate control makes this

approach effective within emergent mobility models,

such as car sharing and on-demand ride services.

Passenger preferences are readily passed between

vehicles as they are carried between vehicles on the

passenger’s mobile device. With on-demand services

such as Uber or Lyft, the commuter's mobile device

can send climate comfort preferences with other

information like location and destination. The driver's

SDL enabled application would modify vehicle

climate control settings to precondition passenger

zone. A remote control feature of SDL allows the

backseat passenger to manage climate settings using

personal device directly paired with the vehicle

electronics.

Other opportunities arise from cloud connectivity

where data collected from sensors and wearables

enables cloud analytics. There is the opportunity for

crowdsourcing of air quality sensor data continuously

from vehicles at many locations to improve real-time

air quality maps. Finally, insights from climate

control usage patterns under different contexts will

SmartDeviceLink Application to Intelligent Climate Control

239

facilitate future advances leading to optimization of

climate control system design.

ACKNOWLEDGEMENTS

We would like to thank the ChemiSense team

(chemisense.co), especially Mr. Charlie Choi and Mr.

Dev Mehta, for their invaluable input in this research

project.

REFERENCES

ChemiSense. 2016. Advanced Air Quality Monitoring.

[ONLINE] Available at: http://chemisense.co/.

[Accessed 26 April 2016].

Daly, S., 2011. Automotive Air Conditioning and Climate

Control Systems. Butterworth-Heinemann, p 432.

AirNow, 2016. Environmental Protection Agency.

[ONLINE] Available at: https://airnow.gov/. [Accessed

26 April 2016].

GENIVI. 2016. SmartDeviceLink. [ONLINE] Available

at: http://projects.genivi.org/smartdevicelink/home/.

[Accessed 26 April 2016].

Kajino, Y., Sugi, H., Kawai, T., Ito, Y., Tateishi, M.,

Samukawa, K., 2000. Development of Automatic

Climate Control with Neural Control. SAE Technical

Paper 2000-01-0978, pp. 1- 6.

Kuang C. W. and Dage, G. A. An Intelligent Automotive

Climate Control System, 1995. IEEE International

Conference on Systems, Man and Cybernetics.

Intelligent Systems for 21st Century, Vol. 4, pp. 2977-

2982.

Lane, N. D., Georgiev, P., Qendro, L., 2015. DeepEar:

Robust Smartphone Audio Sensing in Unconstrained

Acoustic Environments using Deep Learning.

Proceedings of the 2015 ACM International Joint

Conference on Pervasive and Ubiquitous Computing,

pp. 283-294.

Lu, N., Cheng, N., Zhang, N., Shen, X., and Mark, J.,

August 2014. Connected Vehicles: Solutions and

Challenges. IEEE Internet of Things, Vol. 1, No. 4.

Müller, D., Klingelhöfer, D., Uibel, S., Groneberg, D. A.,

2011. Car indoor air pollution - analysis of potential

sources. Journal of Occupational Medicine and

Toxicology, 6:33.

Rosenfeld, A., Azaria, A., Kraus, S., Goldman, C. V.,

Tsimhoni, O., 2015. Adaptive Advice in Automobile

Climate Control Systems. Proceedings of the 2015

International Conference on Autonomous Agents and

Multiagent Systems, (AAMAS), Istanbul, Turkey, May

4-8, 2015, pp. 543-551.

Seraphim Sense. 2016. Angel Sensor. [ONLINE] Available

at: http://angelsensor.com/. [Accessed 26 April 2016].

Shelly, P., 2015. Addressing Challenges in Automotive

Connectivity: Mobile Devices, Technologies, and the

Connected Car. SAE Int. J. Passeng. Cars – Electron.

Electr. Syst. 8(1):161-169.

Smirnov, A., Shilov, N., & Gusikhin, O., 2016. Socio-

cyberphysical system for parking

support. International Journal of Future Computer and

Communication, 5(1), pp. 27-32.

Swan, M., Feb 2015. Connected Car: Quantified Self

becomes Quantified Car. Journal of Sensor and

Actuator Networks, Vol. 4, Issue 1, pp. 2-29.

Thomas, B., Soleimani-Mohseni, M., Jan 2007. Artificial

neural network models for indoor temperature

prediction: investigations in two buildings. Neural

Computing and Applications, Volume 16, Issue 1, pp

81-89.

ICINCO 2016 - 13th International Conference on Informatics in Control, Automation and Robotics

240