Windows 7: The Sensor and Location Platform: Building Context-Aware Applications

1. PC25 Windows 7: The Sensor and Location Platform: Building Context-Aware Applications  Dan Polivy Lead Program Manager Microsoft Corporation

2. What If…? • …your computer knew where youwere and adjusted itself accordingly? • Update local weather, news, events • Automatically adjust clock • …your computer could sense its environment and optimize your experience? • Adjust display backlight based on ambient light • Optimize UI elements for improved readability

3. Agenda • Platform Introduction and Scenarios • Components and Architecture • Location-awareness demos • Light-aware UI demos • Sensor Development Kit Overview

4. demo Location-Aware Weather Gadget

5. Limitations Of Sensors Today • Location devices are exposed as virtual COM ports • Exclusive application access • Not secure • Proprietary data formats (NMEA, others) • GPS doesn’t work indoors • Hard to support multiple technologies at once • Sensors are integrated as vertical solutions • Applications need to know sensorhardware specifics • Limited adoption and scope

6. Windows Sensor And Location Platform • Provides unified driver model for all types of sensor devices • Physical sensors (e.g., GPS devices, Light Sensors) • Logical sensor (e.g., Wi-Fi triangulation resolver) • Provides standard APIs for accessing sensors • Sensor API: C++/COM • Raw access to any sensor • Location API: C++/COM, IDispatch (scriptable) • Abstracted API for location data • Puts user in control of information disclosure

7. Sensor Architecture Application Application Sensor API User Location and Other Sensors Control Panel System Sensor Class Extension UMDF Sensor Driver Sensor Device

8. Location Architecture Gadget or Script Application Application Location API Location IDispatch Interface Sensor API User Location and Other Sensors Control Panel System Sensor Class Extension Sensor Class Extension UMDF Sensor Driver UMDF Sensor Driver Sensor Device Logical Location Sensor (Triangulation)

9. Location Platform Benefits • Single API call to answer “Where am I?” • Independent of provider used todetermine location • Synchronous and Asynchronous models • Script/automation compatible • Automatic transition between providers • Most accurate providers have priority • Concurrent access for multiple applications • Default Location • Provided by user as fallback when no other sources are available

10. Location COM API • interfaceILocation : IUnknown • { • HRESULT RegisterForReport( • ILocationEvents* pEvents, • REFIID reportType, • ULONG dwRequestedReportInterval); • HRESULT GetReport( • REFIID reportType, • ILocationReport** ppLocationReport); • // other functions • }; • interfaceILocationEvents : IUnknown • { • HRESULT OnLocationChanged( • REFIID reportType, • ILocationReport* pNewReport); • HRESULT OnStatusChanged( • REFIID reportType, • LOCATION_REPORT_STATUS newStatus); • };

11. Types Of Location Data • Geographic Data (ILatLongReport) • Latitude, longitude, altitude, associated error required • Most common format • Best format for precise location • Can reverse geo-code later • Civic Address (ICivicAddressReport) • Zip, Country required • Most human readable • Best for ‘rough’ location estimates,street directions

12. Location Reports • interfaceILatLongReport : ILocationReport • { • HRESULT GetLatitude(DOUBLE* latitude); • HRESULT GetLongitude(DOUBLE* longitude); • HRESULT GetAltitude(DOUBLE* altitude); • HRESULT GetAltitudeError(DOUBLE* altitudeError); • HRESULT GetErrorRadius(DOUBLE* errorRadius); • }; • interfaceICivicAddressReport : ILocationReport • { • HRESULT GetAddressLine1(BSTR* pbstrAddress1); • HRESULT GetAddressLine2(BSTR* pbstrAddress2); • HRESULT GetCity(BSTR* pbstrCity); • HRESULT GetStateProvince(BSTR* pbstrState); • HRESULT GetPostalCode(BSTR* pbstrPostalCode); • HRESULT GetCountryRegion(BSTR* pbstrCountry); • };

13. demo Scripting The Location API

14. Privacy And Access Control • Location data is considered PII • User consent is required to share data • All sensors are disabled by default • Admin rights required to enable a sensor • Sensors can be configured on a per-user basis • “Enable Dialog” invoked by applications

15. Location Summary • Single, convenient API for accessing current location • C++/COM • IDispatch (script, .NET interop) • Lat/Long and Address formats Enables applications to be device-agnostic User consent required toaccess data • Opt-in via ‘Enable Dialog’ orControl Panel

16. Working Outside With Windows Vista

17. What Is A Sensor? • Enumerated via category and type • Category represents what is being sensed • Type represents how it is sensed • Properties • Read-only or read-write • Data • Property keys specify data field, units, and type for associated data • Events • Data-driven • State

18. Ambient Light Sensors • Measure light intensity (i.e., illuminance) in LUX (lumens per square meter) • Windows 7 includes class driver support for ACPI light sensors • Working with OEMs to integrate light sensors into notebook designs • Adaptive brightness feature supported by Windows 7 • OS automatically adjusts display backlight • Light-aware applications can use these sensors to optimize UI content for various lighting conditions

19. demo Light-aware UI

20. Light-Aware Applications Photos taken in direct sunlight UI with light-awareness, 100% screen brightness UI without light-awareness, 40% screen brightness

21. Enumerating Light Sensors • #include <sensors.h> • HRESULT hr; • CComPtr<ISensorManager> pSensorManager; • pSensorManager.CoCreateInstance(CLSID_SensorManager); • CComPtr<ISensorCollection> pALSCollection; • CComPtr<ISensor> pALSSensor; • // Get all the ALS sensors on the system • pSensorManager->GetSensorsByType(SENSOR_TYPE_AMBIENT_LIGHT, &pALSCollection); • hr = pSensorManager->RequestPermissions( • 0, // Owner window • pALSCollection, // Collection of sensors requiring permissions • TRUE); // Modal flag • if(SUCCEEDED(hr)) • { • pALSCollection->GetAt(0, &pALSSensor); • }

22. Getting Current Light Level • STDMETHODIMP CALSEventSink::OnDataUpdated( • ISensor* pSensor, ISensorDataReport* pNewData) • { • PROPVARIANT lightLevel; • PropVariantInit(&lightLevel); • // Get the sensor reading from the ISensorDataReport object • pNewData->GetSensorValue(SENSOR_DATA_TYPE_LIGHT_LEVEL_LUX, &lightLevel); • // Extract the float value from the PROPVARIANT object • float luxValue = V_FLOAT(lightLevel); • // Normalize the light sensor data • double lightNormalized = ::pow(luxValue, 0.4) / 100.0; • // Handle UI changes based on the normalized LUX data • // which ranges from 0.0 - 1.0 for a lux range of • // 0 lux to 100,000 lux, this method represents such a • // handler that would be implemented in your application UpdateUI(lightNormalized); • PropVariantClear(&lightLevel); • return S_OK; • }

23. How To Build Light-Aware UI • Scale • Change font size/zoom level • ‘Weight’ of lines • Contrast • Color • Saturation • Complementary vs Adjacent • Smooth transitions between changes are extremely important for overall user experience • Fades or animations • Data smoothing/hysteresis

24. Sensor API Summary • Rich API for accessing raw sensor data • C++/COM • Synchronous and asynchronousdata access • Discovery via category or type Direct access to individual sensors Extensible architecture • Data and properties mappedto PROPERTYKEY andPROPVARIANT pairs

25. Sensor Development Kit • Based on Freescale JM Badge Board (HID) • Sensors • Ambient light sensor • 3D Accelerometer • Dual touch strip sensors • Developer tools • Sample firmware code • Sample driver code • Diagnostic and sample applications • Light-aware MSDN Reader • Marble game Limited quantities available in the Windows 7 booth!

26. Getting Started • Samples • C++/COM samples in Windows SDK • JScript location sample in Windows SDK • WPF/.NET samples in Sensor Development Kit CD • Tools • Light Simulator (SDK) • Hardware • Get a Sensor Development Kit at PDC! • Will be available online in the future(timeline TBD)

27. Call To Action • Utilize the Windows Sensor and Location platform to build location-aware applications, gadgets and services • Implement light-aware UI in your applications to support betterreadability and UX under various environmental conditions • Build environmentally-aware applications using additional sensors with the sensor and location platform

28. Additional Content At PDC • Booth • Windows 7 • Hands On Lab • PCHOL13: Writing a Location-Aware Gadget • Whitepaper • Introducing the Windows Sensor and Location Platform • Implementing Light-Aware UI Using the Windows Sensor and Location Platform

29. Resources • Sensor and Location WHDC sitewww.microsoft.com/whdc/sensors • Windows SDK • Virtual Earth Map gadget (winui\Sidebar\LocateMe.gadget) • Light Simulator and sample • For business inquiries, please contact: • Sensor Platform: sensext@microsoft.com • Location Platform: locext@microsoft.com

30. Evals & Recordings Please fill out your evaluation for this session at: This session will be available as a recording at: www.microsoftpdc.com

31. Q&A Please use the microphones provided

32. © 2008 Microsoft Corporation. All rights reserved. Microsoft, Windows, Windows Vista and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.