Saqib Shaikh: a Blind Engineer at Microsoft develops app that could help blind people navigate in the world

Microsoft engineer Saqib Shaikh, left, standing with Microsoft CEO Satya Nadella lost his sight when he was seven years old. In the pursuit of the freedoms that sighted people all take for granted every day, Shaikh working out of London, UK has been personally involved in the development of an application of Artificial Intelligence, cognitive computing, image recognition and mobile headset technologies.

The image analysis processing, cognitive reasoning and speech intelligence in the device Shaikh uses allows him to ‘see’ the world around him in a way that was considered to be part of science fiction as recently as a decade ago.

Shaikh is shown on a video linked here explaining how the specific confluence of technologies he uses helps him. The intelligence comes from ‘Seeing AI’, which is a research project that helps people who are visually impaired or blind to understand who and what is around them. The app is built using intelligence APIs from Microsoft Cognitive Services.

For example, in a meeting, the “Seeing AI” virtual assistant can describe the apparent moods and level of engagement of the people around the table, giving Shaikh the types of visual cues that he would otherwise be unable to perceive. In a restaurant, the app can help him take a picture of the menu, and then narrate the different options for him.

The app itself runs on smartphones and also on pivothead smartglasses. The glasses are built with a side button that the user touches while wearing to take a snapshot of the world in front of them. The image capture and analysis software that the glasses (or smartphone) uses is able to plug into cloud-based services that will help determine what the user is looking at.

The current state of image analysis software at the time of writing is able to determine the difference between men and women, the shape of standard objects (such as a desk, building, plate of food and so on), the state of facial expressions (such as happy, angry, confused and so on) and whether motion is happening.

A Short Biography

Saqib Shaikh is a software engineer at Microsoft, where he has worked for 10 years. Saqib has developed a variety of Internet-scale services and data pipelines powering Bing, Cortana, Edge, MSN, and various mobile apps. Being blind, Saqib is passionate about accessibility and universal design; he serves as an internal consultant for teams including Windows, Office, Skype, and Visual Studio and has spoken at several international conferences. Saqib has won three Microsoft hackathons in the past year. His current interests focus on the intersection between AI and HCI and the application of technology for social good.