How we do it‎ > ‎

Processing - how we make that useful

An example of augmented vision. A person is identified and represented as a set of red lights.
We can do a wide range of things with the different inputs we have hooked up - our only limits are the processing power we can achieve while keeping the computing module small enough to fit in a pocket, and what our users would find useful. We have picked out a selection of things we could do, using cunning computing methods, and will ask the opinion of our users on which of them are the best.

It could be useful to have:

Brightness to show depth

Having found the depth of the scene the wearer is looking at, we need to break it down into a simple form that they can make sense of. We split the field of vision up into squares and colour each square depending on how far away the closest thing in that square is.

Person identification

By comparing the picture the cameras are picking up now with the one from a few seconds ago, we can tell which objects in the scene are static and which are moving. This isn't a cast iron way of picking out a person from an object - some people stay quite still - but it's reasonably good.

Public transport

Currently, visually impaired people can usually tell there is a bus coming, but not which one. We hope to run image processing on the pictures we receive from the cameras that will pick out bus-shaped objects, find the location of numbers or destinations on them, and use optical character recognition to tell the wearer what bus is approaching. It would also be possible to give the wearer real-time information about how long it would take for their bus to come, as sometimes displayed at bus stops.

Directions

We can use the GPS module to find both the wearer and their destination on a map, and have the computing module pass on directions through spoken words in the headphones.

All of these are possible, and would make it easier for a person with reduced vision to navigate independently and be confident of avoiding obstacles - which they have told us is important to them. However, it is very important that our technology is fast, operating in real time, and reliable. A mistake could get someone hurt.

Cunning computing tricks

Interpreting the signals from the cameras is fairly straightforward if the wearer is still. But we want to help people move, so we have to allow for the fact that the viewpoint is going to move too. This makes figuring out what the cameras are looking at harder - we have to tell the difference between things that appear to be moving because the wearer is, and things that are actually approaching. Changes in perspective can also fox a computer. Our brains have learnt that a bus looks different from different angles, but computers need to be told.

We help ourselves solve these problems by adding information from gyroscopes mounted on the user. These devices tell us how the wearer is moving, and so we can allow for that when we process the camera images. Things that still seem to be moving when we have subtracted what the user is doing are probably actually important things for the wearer to know about.



Comments