How blind astronomers discover things their sighted colleagues missed

Sonification and tactile 3D models not only have made astronomy more inclusive for visually impaired scientists, they’ve also opened the door to more efficient ways of processing data, and even helped sighted astronomers make new discoveries.

By Johnny Bliss

I’ve always been deeply impressed by astronomers and astrophysicists, and their ability to use mathematics to achieve a deeper understanding of the beginnings of the universe. If you could understand all of that stuff and work with it, you can be assured that I was already in awe of your amazing brain.

And that was before I found out about the existence of blind and visually impaired astronomers. Apparently, not only are there many blind and visually impaired astronomers doing good work in the field, in some cases, they are discovering things that their sighted colleagues missed.

Mind officially blown.

Today I will introduce you to multiple astronomers from around the world who have learned how to „stargaze“ without using their eyes. I will also introduce you to a couple of the pioneers who work with them to produce sonification software and tactile 3D models of astrophysical phenomena.

Let’s start with a German astronomer who works at the Karlsruhe Institute of Technology. His name is Gerhard Jaworek and he was born blind at a time when there was very little in the way of such technologies for him to work with. Not that he let that stop him.

Johnny Bliss: Gerhard, what was the initial fascination for you, given that you couldn’t actually see the stars?

Gerhard Jaworek: I’ve been fascinated by astronomy and space since childhood, like most boys. But the door to astronomy opened when I visited an exhibition in Stuttgart of a very large meteorite, and I was allowed to touch it. I thought ‚this is so fascinating!‘, and from this age, I was an astronomer in my heart.

For you, what are the magic moments of astronomy?

For example, the total solar eclipse. I ‚watched‘ it in a park. I had a light detector with me, so I heard the darkness coming and growing, and this was a very special moment. Sonification is a very new field in astronomy. I learned to LISTEN to phenomenae in the sky, for example, solar wind or radio waves from polar lights. There are so many access points to astronomy that the smallest one is to watch the star visually.

Was astronomy always so inclusive?

Traditionally, astronomy has been a very inclusive science for researchers with disabilities. For example, Johannes Kepler, the most prominent astronomer of the last millennium, had a very poor sight, due to a smallpox infection. I am not sure if he could have found his ‘three laws’ if he did not have the assistance of Tycho Brahe, an excellent sighted observer of the sky and data collector. Another astronomer, for example: John Goodricke. Totally deaf. Or Stephen Hawking, you know. Functioning limbs are not necessary to be an astronomer or physicist.

Of course, it’s one thing to be an astronomer, having never been able to see. It’s quite another to lose your sight as an adult, and have to learn to adjust.

Dr. Wanda Diaz Merced is a Puerto Rican astronomer who works at the South African Astronomical Observatory in Cape Town. Some years back, her career suffered a major hiccup when she lost her sight due to a condition called diabetic retinopathy. Far from letting this setback end her career as an astronomer, she instead became a pioneer of sonification.

Johnny Bliss: You were already an astronomer when you lost your vision. Did you despair at being able to continue in this field?

Wanda Diaz Merced: Yes. When I lost my sight, I thought ‚As a disabled person, I don’t have any tools to do my science.‘ I felt that I had been ex-communicated, when I couldn’t see the charts and analyze my data. And then, when I HEARD my first data set and I was able to analyze it, I said ‚Whoa! Life IS possible in this field.‘

What’s an example of something you’ve worked on recently?

I’ve worked on variable stars. We call them variable because their brightness varies. Different periodicities in brightness tell you about different events happening in the stars. We were able to listen to some of these periods - and to us, that was groundbreaking because those periods can be very hard to find, and they were found with sound. But that also showed that when you use sound, your sensitivity to events in the data increases. We’d been missing a lot of information by not using multisensorial methods to explore the data.

So if it had just been a bunch of sighted astronomers this entire time…

...they wouldn’t have seen it! Because they’ve been studying these particular data sets for years! And these events had never been identified.

The word ‚sonification‘ has come up a couple of times in this article already. Now let’s take a closer look at what that means, in practice. (In the most general terms, it means the conversion of data to audio, to convey scale, shape and frequency using only sound. There have been some programs over the years that have been developed to sonify data.)

Assistant Professor Jeffrey Cooke works at Swinburne University’s Centre for Astrophysics and Supercomputing, and works together with a blind PhD student by the name of Dr. Garry Foran to provide sonic data that he can analyze. One example is audio which conveys the spectrum of a distant galaxy.

Johnny Bliss: Garry, you only become an astrophysicist AFTER you lost your vision. Seems like a bit of a weird moment to decide to pursue such a complex field.

Garry Foran: I’d always had a keen lay interest in astronomy and astrophysics. That said, it was not clear at first that it would be something that I was physically able to accomplish. When I first approached Swinburne University and Jeff took me on, we did it on a mutual trial basis, and at the end of that trial period, we came to a mutual agreement that this was something that was going to be feasible. So Swinburne took me on, in the higher degree by research program.

Jeff Cooke: When Garry came to me with his story and his desire to study astrophysics, I did think of astronomy as pretty visual. But I also recognized his passion. So we started to work with data sonification to help Garry hear the spectra. And that worked in really well with a program that I’m working on, which is Deeper Wider Faster.

Deeper Wider Faster?

Jeff Cooke: It’s a very big program that uses about fifty major facilities around the world on every continent, and in space, looking for the fastest bursts in the universe. We run these programs where you get simultaneous data from many of these telescopes. And you need to understand the data, before the fast bursts fade. So you gotta get through a lot of data fast. There’s this urgency there. When Garry came, we were looking for ways to try to analyze the data faster and better, and sound came up as an option. We started learning that there are many areas of sound that haven’t been exploited.

Can you give me an example?

Jeff Cooke: One example is using higher order harmonics of sound. It’s the same thing that helps us discern if we hear a note by a flute, or a trumpet. Same note, same frequency, same duration, same volume. But the human ear can instantly tell that one’s a flute and one’s a trumpet. Different higher order harmonics come off the instrument. And if we can assign different properties to all these higher order harmonics, with a single tone, you can understand maybe ten properties at once. Basically, if you’re going to analyze a lot of data quickly, you want to use everything at your disposal: sight, sound, everything.

In a way, this isn’t even about inclusion. It’s about efficiency.

Jeff Cooke: Well, it’s for everyone! It just so also happens that it serves visually impaired and blind people, allowing them to participate in real science in the community, from anywhere in the world.

Following are some examples of sonification audio, along with written explanations of what you are hearing, courtesy of Jeffrey Hannam, the PhD student at RMIT University in Melbourne who built these tools. (Jeffrey’s research explores BVI experiences of sound within complex environments, and how designers may enhance aural experiences through a sound design approach which considers qualities that shape environment legibility.)

Jeffrey Hannam: The above contains the sonification of two images: 1 medium size galaxy surrounded by 4 stars, followed by 1 large galaxy (hence the initial distortion) surrounded by 6 stars. Sonification playback begins from the center of the image, moving outward. Object circularity is used to distinguish between star (bell sound) and galaxy (rising pitch sound).

Jeffrey Hannam: The tool is set to play other properties of the galaxies (here, mass, star formation rate, and age that are not dependent on galaxy class) and assigns them to higher-order harmonics.

Jeffrey Hannam: Here the tool is set to play both the galaxy class musical instruments and the harmonics. With some training, a person can determine a lot of things about each data point from their single complex notes. To make this more striking, we can add chords to the complexity.

Jeffrey Hannam: The sounds of running the cursor over data points on a scatter plot of data (you move the cursor with the trackpad on your laptop that is scaled to the size of the plot). Here, the data points on the plots are assigned three arbitrary musical instruments for the three types of galaxy classes plotted.

There is, of course, another important way that the blind can participate in astronomy: in a tactile way, through touch.

Cassandra Runyon is a geology professor at the College of Charleston in South Carolina. She is also a pioneer of the use of braille, to bring astronomy, and astronomical phenomena, to blind and visually impaired students. With the help of NASA, she’s recently published a book in braille to reproduce the experience of a solar eclipse.

Cassandra Runyon: In 2017, there was a solar eclipse across North America. It traveled from the west coast to the east coast, and there was nothing to demonstrate or illustrate this for folks with low vision or blind. So we put together three tactile graphics: the first one shows the geometry between the sun, the earth, and the moon. The second graphic goes through the phases of the eclipse, and the third one shows the whole continental US, with the path of totality going from the west coast to off-shore South Carolina, where I actually was on a Coast Guard boat!

Johnny Bliss: What motivates you to produce these books?

I think science should be accessible for everybody. I recently had several students who are blind and deaf, and so what we needed to do for them was different than what I needed to do for the majority of the class. But in the end, absolutely everybody in the class benefited from the adaptations.

Special thanks are due to the folks at „Inspiring Stars“, the IAU exclusive world exhibition, for facilitating these interviews and making this series possible. Thanks also to Jeff Cooke, who patiently explained the ins and outs of sonification technology to me until I finally understood. (It took a while.)

June 17th to 21st: Johnny Bliss spoke with blind and visually-impaired astronomers from all over the world, as well as the people who help them to make their work possible. They discussed sonification, tactile ‘braille’ models, inclusion in astronomy, and much more. This broadcast on FM4’s Reality Check every this week from 12pm, and is streamable seven days on demand.

Check out the Reality Check podcast!