Information selection in a multi sensory manner

I’ve been thinking about touch and
what extra it gives us people. Do we need touch at all? If yes, and most probably we do, why is it so?My thinking
follows like this. We do need tactile
sensation
, it gives an extra dimension to the flow of information we receive. I am not qualified to talk about
the psychology of touch and what biochemical processes are going on but one thing I know for certain is that
looking at a photo of a family member, viewing your lover through a computer screen is nothing like actually
hugging them, holding hands, or even just shake hands with somebody you respect. I could probably scribble
together a paragraph on how oxytocin hormones
are released when executing a touch, but for the moment let’s just assert we require and benefit from tactile
information. It is often easier to grab things and replace them physically than doing it virtually with the help
of a cursor or similar. Often it is proving to be more helpful to explain a topological concept using a 3D shape
rather than a 2D drawing.

If touch has its own specific features we take advantage of, surely all our senses can provide with something
unique, some sort of information, a set of knowledge other senses fail to deliver as quickly, as detailed, as
memorable or deliver at all. Take for instance taste. Even though a food composition
straight out of the fridge might look perfectly fine, tasting it might rapidly reveal the lunch is not as healthy
any longer as anticipated initially. This is probably where taste masters the
most and has the greatest comparative advantage respect to other senses.

As an other example take hearing. It turns out I was
watching a movie with my family a few years ago during Christmas in
the living room. While enjoying the film, we noticed a cracking
type of noise from the kitchen. It was the Advent wreath which
caught fire from one of the four candles. There was no way we could
have noticed it by any other means but hearing, since the partial
wall covered the sight of the Christmas decoration. So hearing gets
a plus there. I could also mention the example of a fire alarm
going off during the night when people are likely to sleep and not
keep an eye on lamps or similar alarming notices.

Let’s shift to smell. Imagine you are listening to music
while cooking and doing some of the washing up. If the oven was
just opposite side to the sink, as it is in my flat at the moment,
you wouldn’t see, nor you would hear anything if the dinner you are
preparing in the hot cupboard would decide to burn. However, you
might be able to smell the smoke and act accordingly. At the same
instance we could probably list similar benefits to smell as I did
considering touch or taste. It feels nice to sniff a pleasing
perfume, as well as it can be alarming of something unhealthy.

Finally the grand master, vision is back. Yet again, vision seems
to be the ultimate sense we use the most, we use the most
efficiently. However, as described above, sight can fail
sometimes. Hence I came to the conclusion we need to segment
information into subsets and assign each subset into the modality
it is best conveyed and processed. Let’s stick to the example of a
scientific graph. For instance a Gauss distribution of data plots
with some scattering and noise. Now let us exclude the most
conventional way of interpreting graphs and diagrams i.e. through
visual perception. Instead I promote looking at the
remaining information modalities. I must admit it is even difficult
for me to imagine tasting or smelling a graph but touching, hearing
and verbalising should ring the bell. Some people would say why
don’t we convert all the information into sounds? Some would ask
why don’t we turn everything into tactile formats?Yet an other
group of people might think, why don’t we describe the graph
verbally?

My point of view is that why wouldn’t we do all of that at the same time. A description could give a nice overview
of what can be seen but nothing guarantees the description gives account on all features a user might want to know
about. So we should give the opportunity to explore. A general trend of the Gaussian curve could be well
demonstrated using a tactile curve additionally to tactile axes. However, considering outliers sound scanning
would be more informative most certainly. Similarly, extracting text labels and importing them into text to speech
should deliver a faster transmission of information than braille cells for example. Thus, splitting a coherent set
of information into subsets and process each of these information bits individually via the appropriate output
modality is something we might need to consider as an idea seriously when trying to overcome a sensory loss.

Standard

what’s wrong with 3D printing?

The title is somewhat misleading I must admit. There is nothing wrong with 3D printing as a way of
additive manufacturing, creating new things, reshaping raw materials. Really, 3D printing is great, me and few of my colleagues actually have a chance to use the technology at a given stage of our research work and admittedly it is a great support to us. However, the question in the title is the one which helped me to understand and find an answer I
was looking for a considerable amount of time. I am heavily involved in a research aiming to develop a
tactile graphics display, which I definitely will introduce at this blog at a later date, but I find it
to be a smarter choice of postponing the introduction until our new website is ready to be published.

While editing our research principles and ideologies page, I was dwelling a lot on how to phrase the
obvious thoughts I, we the research team have, in such a way that it is more or less clear what we want
to achieve, what our objectives are. I tried to retell the speculations around inclusive vs. assistive technology, refer to the multi-modal nature of the device, bring up sensory substitution and also make
sure that not only blind or visually impaired people will benefit from the new technology; but sighted
users are going to feel the advantages of accessing information at a deeper level, using various senses
to perceive, process and interpret the environment around us. Trying to desperately classify the
technology we are about to produce I initially thought I shall make an assimilation with 3D printing.

Probably I would have said something like

“Our technology will teach people how to be less vision dependent and enable a revolutionary way of
accessing information in approaches different from how we see the world recently. The tactile graphics
displays will open up new ways of interacting with information, similarly to 3D printing.”

Instead of writing the idea down and moving on to the next ambitious paragraph detailing how amazing we
are and how fantastic our project is, I thought there are more useful things to do, so I kept thinking
about the forth mentioned statement. It was then I remembered a programme I saw recently on BBC on 3D
printing. Clearly the show made the point that the technology is brilliant, but still what is wrong with
it? Why do we not have 3D printers in every household?

The episode gave a fairly detailed account on the major causes of the slow spread of these devices. Most
of these reasons were the traditional quality-to-price ratio, steep learning curve of operation, the
dirt these devices can make occasionally or the temperature at which they operate. These reasons are
undoubtedly correct but I started to wonder whether there is not a more profound source of not using 3D
printers as widely as we could do.

My response to that insight happens to be somewhat abstract, but what if we are simply not yet ready for
a transition? A transition between viewing graphics in two dimensions on a screen and actually being able to build, grab and touch the three dimensional model of what we only saw on a display a few hours ago. 3D movies
are to some extent are available but still it is not as much of a turning point in media as it possibly
could be. Perhaps our brain is smarter than that. May be we enjoy seeing 3D shapes coming out of the
wide screen in the cinema momentarily, but is it not possible our conscious says “this is not real, just
virtual images with dept but I can’t touch them, I therefore won’t accept it as anything real, no matter
how realistic a scene looks.”? May be we took a too big step because technology allowed us but mentally
speaking we are not quite ready to produce anything that appears on a screen. In my opinion we might
need to introduce an intermediate step between 2D and 3D.

Thus, how I would probably define the device we are about to create, is in terms of a 2.5D display. Why
not give people the opportunity to raise shapes on the screen, let them touch, feel its texture, explore
its tactile response for a few years. Once they got used to the new features, new technology, new
sensation, they will want more, people will be ready to accept more. We always want more… We always
ask for more than what we have but if suddenly we are given too much new information, too many new
abilities, we feel uncomfortable with all the newly acquired knowledge. So a 2.5D display would serve ideally as a transition from visual screens to 3D printed objects. It would ensure comfortable shift toward
touching what we see, satisfy our needs for more and still, give enough time to be prepared for creating
things out of the blue with a technology that is just around the corner.

Standard

Functionality vs. esthetics, Sensory substitution vs. sensory augmentation

As I was reading through the weekly product of some of the mailing lists I am signed up for, a news letter sent by the National Federation of the Blind caught my attention. More precisely a statement quoted from the NFB’s president at a recent talk is what raised my curiosity the most. The assertion goes like this:

Low expectations stand between blind and sighted people, and not the blindness.

Now it is not entirely clear to me what the president of the organisation referred to. Is it the low expectation of blind people respect to the quality of the assistive technology made for the blind community, is it the low expectation of sighted people respect to the abilities of what their non-sighted peers can accomplish, or perhaps it is something completely different.

For the time being, let’s leave prejudice aside, and instead concentrate on technology and the blind point of view as most of the time this blog is going to do. Yesterday I participated in an interview set up by a sensory substitution researcher at the University of Sussex. We had a conversation about a number of things, including life before and after blindness, evaluated 5 different assistive tools, 3 sensory substitution tools they are developing at the moment, and also rated each of these devices in terms of usability. We explored aid utilities all together supporting navigation, everyday tasks such as reading, cooking, communication and of course spent a small amount of time on science accessibility.

Before heading on, here is a BBC Click episode introducing a sensory substitution device being developed in the School of Psychology at the University of Sussex.
BBC Click 23 05 2015 8-12 min

One spot of the interview took a fascinating direction of thoughts. I was trying out a tablet device which converts  colours to sounds. Colour has always been a difficult question when considering blindness. One might ask, just as my interviewer did, how useful such a tablet would be? My response to these type of questions is usually that colour-to-audio transcription has many beneficial aspects for those who are unable to distinguish segments of the visible electromagnetic spectrum. It is enough to think about coloured pie charts or three different curves plotted on a graph with varying colour labels. Colours very often have a significant functionality. We don’t even need to think about abstract concepts such as the pie chart but simply consider the scenario of a man walking down the corridor. It is the colours that help the man to find the door he is looking for. The yellow rectangle within the white ribbon next to him. Of course this is an idealised situation, since in real life we have many clues to help us, such as office labels, handles, door frames and so on. Colours don’t only function as navigation aids in finding certain units standing out of a larger element but colours also have very strong symbolic connotations, for instance national flags, traffic lights, symbolism attributed to black or white etc. Without colours we would lose many of our skills very quickly. However, colours don’t only have functions, they also have esthetic values. Most people tend to agree that some colours are calming, others are depressing, other ones are motivating. I can go further than that, just think about arts, especially visual arts. We don’t appreciate paintings because they are useful but because they are visually appealing.
So when it comes to converting graphics and colours to sounds, we need to ask whether we want to convey the information carried only, just the beauty or both at the same time. Do we want to offer visually impaired, colour blind, non-sighted people only the functional factors of colour or the esthetic component of it as well? Do blind people need the esthetic features? Should they be satisfied with accessing useful information and put aside their other needs? When looking for the answers of the above proposed questions, in fact we are exploring the questions of sensory augmentation and sensory substitution. Do we require the extension of our hearing in a manner that we are going to be able to hear colours as their frequency equivalence and match colour names with pitch values, in which case we are more likely to talk about sensory substitution, or we strive to hear the full range of characteristics a colour can provide, possibly with more conscious understanding than natural sight can serve with, in which case we can consider basic sensory augmentation. This is a matter of choice and expectations on the first place.
The idea of functionality vs. esthetics can be generalised for not only colours but a wide range of other things, for example assistive technology. Do we want speech synthesis to only read a book for us doing the job it was made for, or do we have higher standards and wish the synthetic voice to imitate dialogs, emotions, speak clearly and not like a 60s sci-fi robot?Do we want many devices doing various independent tasks, with no appealing design at all, or we prefer a fashionable gadget which can do multiple tasks adjusted to individual needs? Do we really want to carry braille transcribed magazines as giant books instead of air light paper copies or tablets? Most of these and similar issues are already addressed and with enough attention and effort we can hopefully achieve good progress. However, wouldn’t it be desirable to have non-visual access to advantages a blackboard has, a pen and paper can do, a sketch-pad can offer when it comes to noting equations down in an array, rearranging parts of these equations, sketch graphs on the fly without major time investment in plotting a tactile figure? Do we want to teach disabled kids only the pure and dry facts, mathematical tools or shall we show them all the exciting phenomena in nature, funny and easy to remember, outreach like experiments, demonstrations?
So back to where we started from, the low expectations. Do blind people expect only to be able to have access to science education up to a certain level, enough to be able to learn a certain field of science and become a professional, or is it a fair expectation to ensure a joyful and equally comfortable way of acquiring scientific knowledge and experience?
I shall emphasise at this point that although I have a solid opinion on most of the questions asked here, I don’t necessarily know the correct answers, if there are any at all. Though I don’t have exact solutions yet, sometimes it is more difficult and even more essential to ask the appropriate questions.

Standard

Bad Statistics, Bad Science

I wish I could communicate my thoughts on science as good as the post I came across not long ago does. Though the issue is not related to accessibility of science, I couldn’t possibly find better words why we need to do science and boost its “popularity” at all levels.

In the Dark

I saw an interesting article in Nature the opening paragraph of which reads:

The past few years have seen a slew of announcements of major discoveries in particle astrophysics and cosmology. The list includes faster-than-light neutrinos; dark-matter particles producing γ-rays; X-rays scattering off nuclei underground; and even evidence in the cosmic microwave background for gravitational waves caused by the rapid inflation of the early Universe. Most of these turned out to be false alarms; and in my view, that is the probable fate of the rest.

The piece goes on to berate physicists for being too trigger-happy in claiming discoveries, the BICEP2 fiasco being a prime example. I agree that this is a problem, but it goes fare beyond physics. In fact its endemic throughout science. A major cause of it is abuse of statistical reasoning.

Anyway, I thought I’d take the opportunity to re-iterate why I statistics and statistical…

View original post 1,625 more words

Standard

Why not doing research all the time can make you a better researcher

There we go, I figured out how to reblog I actually wanted to but failed doing so in my “Research career” notice.

In the Dark

Yesterday I read a nice little article in Nature about how doing something different from research every now and again can actually make you a better researcher. I agree with that completely, so thought I’d expand upon the theme with a few comments of my own. I think this is an issue of particular importance for early career researchers, as that is the stage at which good habits need to be established, so I will focus on PhD students.

The point is that a postgraduate research degree is very different from a programme of undergraduate study. For one thing, as a research student you are expected to work on your own a great deal of the time. That’s because nobody else will be doing precisely the same project so, although other students will help you out with some things, you’re not trying to solve the same problems as your peers…

View original post 1,083 more words

Standard

Research career

I am wondering more and more often in the recent times whether I want to do “pure physics” as some would say and leave science education and accessibility to others, whether I should do both scientific research and support SE, or occasionally I have a feeling I am not competent enough to do hard science therefore I should first and foremost focus on making science inclusive, only then start my own pure science life. Posts like this, help for youngsters like me at the beginning of their science career.
PS: I am trying to reblog a post, so if I don’t succeed and only my short post appears, I will try again.

Standard

Confession

I need to confess as I might be doing more spelling, grammar and typing mistakes than it is acceptable at an online diary. I am going to keep applying censorship on my own words, correct errors when spotting them and also I would like to maintain a regular posting style, work permitted though I am afraid. Having said that, I wish the content is going to compensate the amount of language mistakes.

I will also strive to include more links in the future on things I am mentioning in a particular post. Thank you for your understanding and interest.

Standard