A scrambled breakfast

Dear Debriefers,

While dragging myself awake, I made scrambled eggs for breakfast. I cracked the egg, splashed some milk, sprinkled salt and pepper into the pan and stirred.

The concoction smelt tangy, but I was pretty sure the milk was new. I tipped the eggs onto a plate, noting that it wasn’t as fluffy as usual.

And then I started eating. I gagged, as iron, salt, egg and pepper washed my tastebuds. I walked to the fridge to double-check the milk. I sniffed it and realised I’d put pineapple juice in my eggs.

The shape of the carton was slim and rectangular, square top with a cap on the corner. Exactly the same as the milk beside it. My accidental experiment with breakfast happened over ten years ago.

In the decade since then, artificial intelligence has radically changed possibilities for access to the visual world.

This article is about the opportunities this technology brings, but also the way it shifts responsibility from society at large to individuals expected to keep up with the latest developments.

Ria Andriani is a writer and musician living in the Blue Mountains, Australia.

Kinanty Andini is an illustrator and digital artist, from Indonesia.

Original writing like this is made possible by support from readers. With thanks to Bill and Gordon for new contributions.

A better life as a blind person

I was born with congenital glaucoma, and lost most of my sight when I was five. I was born in Indonesia and then migrated to Australia as a teenager.

I was sixteen and a half when I arrived in Sydney. I hardly spoke any English. I left all my friends behind. And for months, I didn’t attend school.

Among the many reasons why my mum took the plunge to move countries was to give me a better life as a blind person.

Yet she was always caught between the desire to give me the necessary tools to ensure I can live as a “normal” life as possible and wishing I could somehow acquire some sight.

Despite the impossibility of the latter, it remains her dearest wish.

In turn, I swung between learning the skills to live without sight and somehow wishing there was a way I could navigate the intensely visual world.

By my early twenties, I was still living at my parents’ home, a place where I couldn’t fully make adjustments to cater for the fact I am blind and have a print disability.

Apps can solve all your problems

It was the 2010s, an age when the prevailing mindset was that apps could solve all your problems.

The first visual aid app that burst into my consciousness was TapTapSee. Released in 2012, it promised to help blind users to identify objects by taking photos and sending them to a cloud-based image-recognition database.

For a time, there was a team of volunteers who would describe the photos sent by users, with the caveat that we must promise never to submit any lewd or prohibited content.

The technology could be really helpful on an individual level.

Within a few years after my breakfast experiment, it would be technically possible to differentiate between milk and pineapple juice, provided I had the right phone, used the right app and can take a good enough photo for identification.

So why didn’t I jump on the technology?

Asking for help as an admission of failure

Before moving to Australia, I lived with my father and his wife in one of Indonesia’s big cities in Java.

They often told me that one day, I’d have no one to call for help. I had to learn to be an ideal woman i.e.: someone who can take care of a house, husband and kids without technology or any additional aid.

They told me that other blind people could live independently. Older blind women who were supposed to be my role models agreed. In my parents' world, asking for any help amounted to an admission of failure.

It coloured my attitude towards technology. Using something like TapTapSee, especially during the volunteer-run phase, would mean outsourcing my inability to cope. When people stopped volunteering, or the program got pulled, I’d be back to zero.

As an adult, I have a very different mindset. I think it’s one of the most unhelpful narratives, one that stopped me from learning how to build interdependent relationships with communities and friends.

Transcending visual culture

The technological development picked up pace while I spent my 20s finishing my degrees, navigating the job market, travelling overseas and figuring out how to adult in my own terms.

I eschewed bespoke adaptive technology, like a water-level indicator to pour tea. But I quickly become comfortable with smart technology like the iPhone.

And then in the middle of 2022, ChatGPT and other AI-powered technology came on the scene. I first heard about BeMyAI through social media. One friend after another posted how the app’s image description generator revolutionised their lives.

One poster described downloading photos from Facebook and submitting them through the generator to find out its contents. Technology podcaster Jonathan Mosen demonstrated how BeMyAI described his family holiday photos, recreating memories of his visit to the Tower of London.

Other friends experimented on personal and environmental photos.

Before long, the phrase “the image shows a …” followed by a lengthy description became a refrain on social media. For a hot minute, it felt like we had transcended the visual culture.

Where it comes from

The release was the result of years of development, data gathering, and innovation. And one detail really caught my attention: AI learning used accessibility features such as captions and alt-text to improve image recognition and generation.

Features such as alternative text, or alt-text as is more commonly known, is a way to tag images with descriptive words, something the blind community has been advocating for years to improve accessibility. Those word tags enable AI to recognise images and generate new ones through prompts.

On social media, most blind users have made a point about how inaccessible photo posts are, and would you please, please, please do this extra step of describing it. We applauded those who did and ignored people who never bothered.

Slowly sliding in

With the arrival of this technology, we were in a brave new world. Thanks to AI, describing an image would be as easy as drinking a glass of milk.

I slid into the apps slowly, feeding it titbits of non-human objects. A piece of clothing, photos of my guide dogs, a point-by-point analysis of the colour difference between my geraniums.

I read the generated descriptions avidly, waiting for it to make mistakes. When it did, it was subtle. And I couldn’t tell if the mistakes were my bad photography or something else at play.

The pull of “normal” might go too far

Yet I couldn’t stop feeding it photos of my patterned coasters, painted mugs and commemorative spoons. I felt beguiled by the benefits, but my reservations of the risks didn’t disappear.

Where would these photos go? What would the app learn about me, my life, where the dog bed was in relation to the door, how I liked to arrange flowers?

And while reading the description was fun, at times I felt the pull of “normal” might go too far.

I’ve worked really hard to claim my body image. After growing up in a household that regarded mismatched outfit as a sin, I spent years trying to define my own style. It took years for me to know my body better than anyone because I live and breathe in it.

Since AI became part of my life, I’ve used it to coordinate outfits, and I’ve spent more time choosing what to wear than I did in the past. Do I just want to look good, or am I trying to live up to my family’s idea of transcending disability?

This tacky thing around my head

Another revolution was also happening: that of smart-glasses. These are meant to provide a hands-free experience for users.

Combined with the use of apps like BeMyEyes or AiRa – which also offer human descriptions – you can put on the glasses and listen to someone on the other side of the world as they become your eyes.

I remember these wearables as big, bandana-like pieces with straps that proclaim: “I am blind, therefore I wear this tacky thing around my head!”. I wouldn’t wear them.

Nowadays, you can get Ray-Ban Meta glasses and connect to a volunteer through BeMyEyes, and AiRa has its own patented glasses.

I wonder about the implications towards broader society, such as covert filming becoming easier.

Accessibility and consent

In the ever-shifting goalposts of “normal”, using a smart phone to solve most everyday issues with apps now seems to be expected and also common-sense.

Still, I’d rather not send photos of my friends or personal information to these description generators. In my mind, doing so blurs the line of accessibility and consent.

Recently, a friend offered to send me a photo of a cherished baby, who had passed away. I couldn’t stomach the idea of this precious photo in the maw of the machine that generates the descriptions. Instead, I listened as my friend described what the baby looked like.

Opening up my history and navigating the future

It’s no longer hard to wave my phone and use an app like Seeing AI to read print. I use it almost daily, from reading food labels to washing instructions.

Over the last few months, I’ve also been able to independently identify the content of my storage boxes: from old university papers to the titles of printed music scores.

By switching my Voiceover language to Indonesian, and its capability to decipher handwriting, I could even read my old school reports.

I’ve used a combination of BeMyAI and AiRa to reorganise my wardrobe. I decide whether to keep clothes or donate them to charity after reading descriptions of what they look like.

On social media, these apps help me to assess second-hand furniture on the local buy-nothing group; or savour the small details in meteorological photography.

Onus on individuals to meet our needs

The onus is on blind users like me to meet our own needs for accessibility. The arrangement works as long as I’m happy to download the individual photos and upload them to the AI image description generators.

But AI doesn’t always get it right. A friend posted a photo of a cake that Apple’s automated image detector said was “decorated with mushy peas”. (The friend later clarified they were M&Ms). Another time, a friend’s roast lamb was pronounced as “a photo of a pangolin”.

We all seem to rely on the technology to solve the problem of access and inclusion. But making the world more inclusive and accessible is a responsibility that should be shared by everyone. Now that AI has made some things easier, companies and people are abdicating their part.

On my social media feed, the number of photos with good-quality alt-texts haven’t increased. And it hasn't become easier to add captions or alt-texts while creating content on Facebook, Instagram or TikTok.

Our bias poured into the tech we create

Once we move past the benefit and risks, there’s one word that haunts many disability users in the AI-revolution: bias. Biases in language, or ideas about what inclusion should be.

There’s a need for guardrails against replicating the biases which have locked us out of most existing systems. At the serious end of this spectrum is whether, for example, AI is given decision over a blind person’s employability.

I recently read a story of Freedom Scientific’s Picture Smart AI erasing the word “blind” from a guide Connor Scott Gardner wrote for blind students to navigate university. Gardner reminds us that “our biases are poured into the tech we create”.

Back in the fridge

I’ve come to regard AI as a tool, not the great disrupter as many first think. Having it at my disposal has been an incredible ride. But it hasn’t made all my problems disappear overnight.

Many of those problems were solved by understanding how I function as a blind person.

Back in the fridge, I make a point of stocking milk bottles that are different to the juice bottles. In addition to waving my phone around, I’ve developed a labelling and organisation system that mostly works for me.

I still have the occasional cooking messes, like pouring a can of condensed milk onto a gado-gado salad, thinking it was the sauce. I just rinsed off the veggies and started again.

Written, without artificial intelligence, in the land of the Dharug People of Australia,

Ria

Outro

Further reading. See the Debrief Library for a collection of resources on disability and artificial intelligence.

See more from Ria, who writes about music for ABC News and is on Facebook.

See more from Kinanty: who is also on Linkedin.

Let your friends know. Sharing the newsletter is how people find it!

Connect. Get in touch, and find Disability Debrief on Linkedin.

Help us do more. Make a one-off or recurring financial contribution to the Debrief.

Acknowledgements

Thanks to Kinanty Andini for the illustration.

I’d like to thank my fellow blind writers Connor Scott Gardner and Ariel Silverman for their research and insights in the development of AI.

I’m also grateful for the community of ABC Inclusive, a disability-lead network group and is a force for change in the Australian media. You really embody our moto: nothing about us without us.

Lastly, I’d like to say a huge thank you to Peter Torres Fremlin, who edited this piece, and the organisations and readers that support the Debrief