Autism is a complex disorder. For me to address the issue in a college environment required a ton of research. I began by conducting interviews with people on campus who deal with disabilities. I spoke to people who work with people having autism to gather insights into their work process. Along with the interviews, I read some books on disability like "Design meets Disability". I read through multiple research papers. I mapped out the information I gathered into Literature Maps to keep track of important information.

I did an analysis of different products that are available in the market. I needed to understand the current solutions available. What makes those products work, what is the pain point that's they are addressing, and so on. I also looked into the latest technology to understand the scope and practicality of using the tech.

I examined different theoretical frameworks to understand the functioning of the system. This allowed for a stronger knowledge base. This helped me develop a blueprint to support the proposed designed system.

Using Deep Reinforcement Learning, the ML model learns how to respond and interact with people on the spectrum creating a training dataset. The dataset exists on an online cloud. Initially, the mediating device uses this repository to respond to the user. Every time the ML model responds correctly, it gets a positive reward. The more the individual uses the system, the more accurately it will learn to respond. The model learns from the sensor data and the human’s response. This allows the modification of the dataset, making it unique for that specific user.

Executive functions are a set of cognitive skills that help us perform different tasks like time management (McKeon, Alpern, & Zager, et al 2013). For those on the spectrum, certain executive functions are not as strong, making tasks difficult. Sensors connected to the user (smartwatches or phones, for instance) can track the users’ abilities and behavior. The sensor data is used to update the dataset to increase the ML model’s accuracy. The more the sensor data sent by the user, the more articulate the results will be.

I'd be lying if I said the process was simple or linear. I learned a lot during the making process. For instance, while creating hi-fi wireframes, I came across certain details that needed revision. I even had to go back to editing the conversation design and the forms. The visuals also changed with the narrative. The designing process helped the project evolve.

User Journey Map

Based on literature research and interviews, I developed 3 different user personas to capture the diversity of people with autism. However, it still does not represent the holistic view of people with this disability. I mapped out their experiences based on their different abilities and preferences. Using these experiences, I was able to find pain points and begin my explorations for the design of the system.

Conversation User Interface

For the project to move forward, there needed to be a medium or interface through which the computer could converse with the user. Thus, the need for Conversational User Interface (CUI). I delved into learning and developing the CUI system, the quality of the relationship was critical to its success. How the person perceives the AI would affect the quality of the mentor relationship. I investigated the personality traits that are sought after in a mentor and worked on designing an AI system embodying those characteristics. The explorations ranged from form to language.

Avatar forms ranged from human avatars, anthropomorphic characters, and abstract forms. After a series of iterations, abstract forms cast a closer representation of the AI personality.

Discerning the depth and complexity of a conversation required mapping conversations between the user and the AI system. The user’s personality and the mode of conversation were critical to the nature of the conversation. Utilizing a range of software such as Voiceflow, uncovered different trajectories and possibilities for the flow of the conversation. This exploration allowed for refining sentences, word choices, and approaches.

Storyboard

By creating storyboards I was able to delve deeper into the intricate details and have a greater understanding of the situation.

UI/UX components

This ranged from designing the UI elements, wireframes and prototyping to best translate the experience of using the proposed system.

Deliverable

To begin investigating this problem space, I needed to design an AI system that would take up the role of a mentor. I named the system, Smart Adaptive Mentor (S.A.M.) and works by using ML (deep reinforcement learning to be more specific). The design of S.A.M. is vital for the development of the project. S.A.M. is an intelligent piece of software that can integrate with your smart devices. It is not platform specific, increasing its accessibility. A knowledge base is created using data collected from therapists, mentors, parents, and others who work with people diagnosed with autism. The knowledge base is used as a training model for S.A.M. It initially relies on this training set to determine its interactions with the user. The more the user interacts with S.A.M., the machine learning model can learn and modify the dataset for that specific user thus creating a unique experience. S.A.M. gets information from connected sensors, devices, and software that the user provides access to. To help S.A.M. be more transparent and communicate with the user, S.A.M. has 3 main states apart from the inactive state (or logo).

To better understand S.A.M.’s abilities, we will be looking at 3 cases of 3 vastly different personas. While I did design other interactions to show how the system builds its relationship with the user. I'll be going over one interaction for each of my personas. To see the other interactions you can check it out my document.

Persona 1: Ezra Doyle

Persona 2: Anya Rathore

Persona 3: Evan Stevens