Methods in Human Computer Interaction/Qualitative/An ethnographic research study of the deaf community in the workplace

The purpose of this study is to understand how the deaf and hearing impaired community uses interactive communication technology (ICT) to communicate with the non-deaf community focusing on workplaces. There are still huge gaps and barriers of communication methods among the deaf community, so they have less opportunities to be involved in our community, society and group. This study addresses the current situation of the user community and population in the deaf, introduces case studies and products introducing deaf communication methods, and develops research methods of ethnography for UX and UI design from existing ICTs.

One of the latest technologies related to this study is “Motion Savvy UNI,” which is leap motion communication for 1st sign language to voice system. UNI using gesture and voice recognition technology in the Motion Savvy provides two-way communication tool for the deaf and hearing. The MotionSavvy software leverages the Leap’s 3D motion recognition, which detects when a person is using ASL and converts it to text or voice. The software also has voice recognition through the tablet’s mic, which allows a hearing person to respond with voice to the person signing. It then converts their voice into text, which the hearing-impaired receiver can understand. The device shows the motion capture through the screen while users communicate with people by sign language. However, there is not yet any verifying accuracy by users (the deaf) whether it interprets accurately or not since they cannot hear voice. This study includes methods of usability test for employed deaf people in workplace who are dealing with verbal communication with non-deaf people. The research method approaches ethnography in terms of benefits of findings and measurements of qualitative means.

User Community, Sample Population

One of the biggest associations in the United States for the deaf community is ‘National Association of the Deaf’ (NAD) and 50 states organized by affiliate members of the NAD as independent nonprofit organizations in their own right [1]. According to the report from the employment statistics of the deaf in the U.S. in 2013, the number of employed is 2, 080,228 based on the total deaf population of 4,140,213 in 2013. It is 50.2% in national average rate while Wyoming (WY) shows 70% as the biggest rate vs. Puerto Rico (PR) recorded 28.7% at the lowest rate.

American Sign Language (ASL) is the predominant sign language of deaf communities in the United States. There is not yet accurate account of ASL users, but estimated number of ASL users including the deaf and non-deaf ranges between 250,000 to 500,000. The survey did not distinguish between ASL and other forms of signing; in fact, the name “ASL” was not yet widespread use [4]. While current technology services such as assistive listening, hearing aids, captioning for access, video remote interpreting provide to the deaf, the NAD advocates for and looks forward to an even brighter future where new technologies take root and tumble communication barriers to ensure equal access for deaf and hard of hearing people and full participation in all aspects of American life. At the same time, the NAD seeks to ensure that new technologies, applications, and equipment are accessible, available, and affordable to people who are deaf or hard of hearing.

The Sprint National Traditional Relay Service introduced ‘State/711 Relay Service’ that a person who is deaf, hard-of-hearing, deaf-blind, or speech-disabled uses a TTY [7] to type his/her conversation to the relay operator who then reads the typed conversation to a hearing person. The Relay Operator relays the hearing person's spoken words by typing them back to the TTY user [6].

Another data of survey which gathered information of employment data for Deaf adults addresses; They don’t feel that they have as many chances for promotion, to work their way up, or take on greater responsibility at work in terms of communication barriers. However, deaf young adults are employed at higher rates than their peers with other disabilities [5].

- Deaf employees are exposed to more misunderstandings than necessary, more underemployment than we should tolerate, and fewer opportunities for advancement than their hearing peers. They often miss communications entirely. Soft skills—which rely heavily on the ability to hear and communicate in English—are a greater predictor of job satisfaction and performance than the more concrete technical or academic skills required on a job. Soft skills include: observing forms of etiquette that are expected in hearing culture; using humor appropriately; working well with a team; problem solving with coworkers; and much more. [8]

- Employers and coworkers often incorrectly place the blame on ineptitude. Unfortunately, deaf people are frequently unaware of these issues so they are not able to advocate adequately for their needs. For those deaf individuals who are capable of identifying and resolving accessibility issues, though, the ongoing communication challenges get in the way.

Methodology

Ethnography

This study uses the ethnography research method to evaluate and to understand how the deaf and hearing-impaired community uses interactive communication technology (ICT) to communicate with the non-deaf community and focuses on interactions in the workplace. Ethnography has been defined as “the art and science of describing a human group – its institutions, interpersonal behaviors, material productions and beliefs” [11]. This research method attempts to gather a deep understanding of an unfamiliar sample population. There have been many studies in the past 10-20 years focused on developing interactive communication technologies for the deaf and hearing-impaired but most, if not all of these studies, fail to focus on how the deaf and hearing-impaired use these technologies in the workplace to communicate with non-deaf people. Issues such as accuracy, quality, efficiency and usability are rarely addressed and a long, focused ethnography of the deaf and hearing-impaired communicating and using ICT in the workplace is nonexistent.

Protocol (step by step)

Selecting a group of interest

First, we selected our target sample population. The sample population used in this study is anyone of the deaf and hearing-impaired community with a job requiring them to communicate with non-deaf individuals. Our study focuses on how this sample population uses ICT in the workplace. By having a general sample population it allows for a broader pool when selecting participants. It also allows the researchers to be exposed to and observe the deaf and hearing-impaired in many different types of work environments.

Choosing a participation role

Participation is a critical part of ethnography but there are limits to what can be learned from the two extremes of participation: complete participation and complete observation. Complete participation means the researcher joins the group in order to study the group, sometimes unbeknownst to the participants. While complete observation means the researcher observes the group without participating, even it situations that may be confusing. Because this study takes place in private workplace environments and requires that the individuals involved know that they are being studied, the best way to collect detailed data is by allowing the researcher to observe interactions but also ask questions and participate in situations that are not clear. In order to accomplish this, the researchers in this study avoid these two participation extremes, and opt for this intermediate approach.

Building relationships

Ideally, all the deaf and hearing-impaired individuals participating in this study as well as their colleagues would warmly welcome the researchers into their workspace. This is not always the case, specifically in workplace ethnographies, there may be concerns about using the research against the participants or that the research will interfere with the participants work. Careful attention to a few principles helps ease these concerns during the study and help portray the researchers as someone to be trusted. First, the researchers make sure to take the time to explain why they are there, what they hope to learn, and what they hope to do with the data they collect. Second, the researcher always respects the needs and goals of the individuals they are studying and lastly, the researcher never puts their needs for data, before the needs and privacy of the participant.

Data Collection Methods

Ethnographic researchers have developed a variety of theoretical frameworks to inform their investigations [11]. These frameworks have been designed to provide perspective on how groups function, on human interactions, and human relationships. Ethnographic studies rely on a broad range of data collection techniques such as observation, interviews, and documents or other artifacts as their primary source of data. The goal is to use triangulation and corroboration of evidence from multiple perspectives to increase confidence and validity of results. This case study relies on these three data collection techniques in order to increase the validity of its results.

Interviews

Interviews in ethnographical research are unlike traditional interviews where a researcher has a single meeting with a participant for a short period of time. Interviews in ethnography are generally part of a longer, ongoing relationship with the participant; a much richer set of data can be collected through this process. This also means that the early stages of the interview process can be less formal. Interviews are aimed at building a relationship and trust with the participant rather than collecting detail information or data. In this study, in this study that means that the non-deaf researcher and the deaf or hearing-impaired participant need to develop a method of communication even before the informal or formal interviews can take place. This will look different depending on the comfort level, skills, and preferences of the deaf participant. In some cases an ASL interpreter will be used, in other cases ICT, or another method such as written communication. Once a communication method is determined and the researcher and participant have gotten comfortable communicating, formal and informal interviews take place.

During the interview process, probes, appropriately focused questions and leading questions are used to get a better understanding of the communication processes that happen within the workplace between deaf and non-deaf individuals. Probes are used in this study and are items that are designed to provoke reactions from the participants and spark conversations. Focused questions are used when a participant is describing a situation and the researcher asks, “how often this situation occur?” or “tell me more about what lead to this situation”. Leading questions are used in this study to invite agreement or disagreement to a viewpoint. More formal interview techniques are also used and include life histories, and time diaries. Life histories help the researcher get a better understand of where the participant is coming from and what their life experiences have been like. While the time diaries help the researcher understand what the participant spends their time on and how that could be affecting their communication methods.

Observations

People are not objective observers, we filter what we see and hear, and interpret our observations though the lens of our own experiences, expertise, and bias. Ethnographic observations hopes to shed this bias and see things with a new lens. If a researcher only documents what they see they may risk misinterpreting or injecting bias. Instead, the researchers in this study interpret what they see as it happens. For example, the researcher would take a note like this: “the participant became frustrated when he wasn’t able to understand his colleague while they were communicating with an ICT ” instead of this “the participant wasn’t able to understand his colleague using an ICT”. One of the other challenges with ethnographic observation is taking appropriate notes and knowing how much detail is really necessary. To help avoid becoming overwhelmed with the quantity of data the researchers selected a few main goals for their data collection. Documentation and note taking were limited to these specific areas, helping to establish focus on the appropriate data.

Documents

Documents, archives and artifacts can be a helpful source of data and information in ethnographies. Documents can help provide insight into how a group of people communicate and work together. In workplace environments there are many sources of documents such as emails, performance documents, and tools. The collection of any documents, archives, or artifacts that can help describe how deaf and hearing-impaired people communicate is identified and collected in this study. This includes documents that help describe how communication has changed for the deaf or hearing-impaired participant over time.

Qualitative Analysis

Grounded Theory Approach

The grounded theory approach will be used for the qualitative analysis of the study. This type of approach works well for ethnographic studies as it allows the data collection and data analysis to have continuous interplay and be iterative. The grounded theory approach requires the researcher to approach the study with no preconceived notion of the theory. Then data is collected and analyzed over and over in an iterative cycle to form a theory and potential solution[14]. The grounded theory approach does not have a standard protocol, but instead requires the researchers to be creative and have an open mind while collecting data.

Subjective coders

Since this ethnographic study is of such a specialized subject matter, subjective coders will be used. The risk of using objective coders, and the coders not completely understand the nuances of the data, is too high. Instead, the researchers will complete the analysis and coding, therefore becoming subjective coders. This is a benefit, because they know the research and literature well, and can understand the concepts collected in the data. However, the researchers will need to be careful in the analysis to think beyond any concepts they have already established in their mind during the research[14].

Open Coding

Initial coding will be done using priori coding. Priori coding involves researchers to determine a few concepts up front for all coders to use[14]. Sample concepts that will be used include: hard skills, soft skills, using humor appropriately, limited advancement opportunities, and problem solving with coworkers. Following the initial concept creation, emergent coding and in vivo coding will be used. These methods will develop additional concepts based on the responses of the community in the interviews.

Conceptual categories

Once concept coding is complete, the coders will begin to develop conceptual categories that will categorize the data together even further[14].
Sample categories include:
Objectives: use motion savvy UNI for work purposes
Causes: Not hearing office chatter
Strategies: multimodal interaction
Consequences: limited opportunity for promotion
Context: deaf users are knowledgeable, just limited by hearing.

Reliability

One important factor in ethnographic research is to make sure the data is reliable and have consistency in the coding. To do this, multiple coders are used, and they all need to code the data the same way. To ensure consistency, inter-coder reliability is used. This means multiple coders with varied backgrounds or theoretical perspectives will be used to code. If they arrive at similar coding conclusions, the results are reliable[14].
To achieve this, researchers must develop explicit coding instructions from beginning of the process. All coders must be trained on the coding instructions and have a complete understanding of the process. Then all the coders will test code the same data. If desired reliability level is reached, actual coding can begin. If not, all coders need to be re-trained and re-tested until desired reliability level is met. This will ensure reliable coding results[14].

Validity

An important factor in ethnographic research studies is to determine the study’s validity. For the study to be valid, it should:
• Fit the phenomenon and be relevant to that area of study
• It should provide understanding of the deaf/hearing relationship in the workplace
• Data should provide a generality without too much variation
Study is considered valid and reliable when no new concepts are developed when new data is collected[14].

Method Justification

McGrath's 3-Horned Dilemma: Where Ethnography Fits

McGrath's 3-Horned Dilemma is a model used to characterize research across 3 characteristics:

Generalizability over populations
Precision of variable control
Realism of observational context

[13]

The "tugging" (like a horn) of one of these characteristics is meant to imply a trade-off of the other two. Since ethnography maximizes the realism of observational context, it greatly reduces the levels of generalizability over populations and precision of variable control:

Low generalizability: difficult to extrapolate results of research to other groups outside the population studied.
Low precision of variable control: since things are "just happening" with little to no control over factors that impact their behavior.

It's also important to note the "ripple effect" of these two costs: since we cannot control the factors within an ethnographic study, we cannot attribute causality between factors and therefore cannot generalize results to other groups.

Comparing Other Methods: Why Not a Survey?

There are various reasons why a survey would not be as effective as ethnography for this study:

A survey would achieve breadth but suffer from a lack of depth.
There is a loss of context with surveys and hence participants are forced to recall information from memory which may be unreliable.
Surveys are incongruous with the goal of the study which is to conduct a needs assessment, which requires understanding the culture, context and group interactions of a community. [14]
There is a greater chance of collecting incomplete and inaccurate data with surveys since:

- Many of the questions in a survey are fixed which leaves room for incomplete data.
- Most participants on surveys are left to their own devices and do not elaborate on their responses for open-ended questions, leaving out "the why" behind answers.
- There is no way to efficiently ask follow-up questions from surveys.
- Fixed answer choices force selection bias since there might be other choices that a respondent would pick, but are left out for one reason or another.

While there still may be forced recall using ethnographic methods (like interviews), inaccurate/incomplete info can be mitigated by having participants actually do the things "in situ" they're describing eliminating the need to rely on memories. Ethnographic methods allow for follow-up questions, open-ended discussions, demonstrations, and participation by the researcher to encourage empathy and a deeper understanding of the problems faced by the group. Surveys look at high-level trends across a given set of group(s) and often look to achieve statistical confidence in data. Ethnographic studies look at "lower-level" specific behaviors, customs and challenges within a group in order to uncover a set of needs; statistical confidence is irrelevant.

Benefits of Ethnography

The following are 3 benefits to conducting ethnography:

1. More holistic method affords greater depth and insight into:

Behaviors, attitudes, motivations and beliefs of a group that would be difficult to impossible to obtain through many other qualitative means.
Allows researchers to probe and respond with follow-up questions that cannot be done with some other qualitative methods (like surveys).

2. Achieve validity through triangulation of methods:

Ethnography employs multiple methods allowing researchers to triangulate on data, improving its validity and reliability.

(ex: the use of multiple informants to validate information)

3. Helps establish a "beachhead" and foundation of research to inform future directions:

Implications for design or follow-up research on a specific technology or product
Understand system requirements and user needs.
It is an inductive process [14] and allows for the construction of descriptive models theories based on the insights and observations from the study.
Helps to validate or refute existing research on the subject.

Disadvantages of Ethnography

The following are 3 disadvantages of ethnography:

1. Expensive to implement in terms of both time and money:

Studies can go for months or even years
Large amounts of qualitative data requires transcription of interviews, observations and coding of unstructured data

2. Cannot necessarily generalize results to other groups

Little to no control over variables
Dealing with "a sea"of confounding variables
Unable to attribute causality between factors
Reference results from research with care (consider caveats)

3. A challenging method to implement:

Requires great skills in deciphering between observation and interpretation
Dealing with contradictory, incomplete and potentially biased data.
Dealing with a group that speaks a different language than the researcher requires an interpreter which may create opportunities for observations to get "lost in translation."

What Does It Mean to the Research Question, Stakeholders, HCI Community, Deaf Community?

...to the research question

Greater certainty around the scope of the needs of the deaf community:
A better understanding of their culture and context reveals unexpected and unspoken answers to key issues and concerns.
Establishes focus areas from which design implications may flow.
Understand which future questions to ask for further research.

...to the stakeholders

Insights and findings become translated into system and user requirements:
- A sound basis for designing communication technologies for deaf communities.
- Assist in prioritizing feature-sets for deaf community.

...to the HCI community

Informs which variables and factors to consider for future research related to deaf communities, both qualitative and quantitative:
- Which variables to consider controlling for/be mindful of for experimental research.
- Which factors to consider for surveys, field studies.

...to both stakeholders and the HCI community

Help formulate a set of principles and guidelines for future products.
- "Do's" and "don'ts" to save companies and researchers from making costly design and development mistakes.

...to the deaf community

Provides greater opportunities:
- More equal footing with the non-deaf community.
- May open doors for other career considerations where communication is a critical component for employment.

Resources

1. National Association of the Deaf: <http://nad.org>

2. American Sign Language: <http://en.wikipedia.org/wiki/American_Sign_Language>

3. Mitchell, Ross; Young, Travas; Bachleda, Bellamie; Karchmer, Michael (2006). "How Many People Use ASL in the United States?: Why Estimates Need Updating". Sign Language Studies (Gallaudet University Press.) 6 (3). ISSN 0302-1475.

4. Employment Statistics of the Deaf in the U.S. 2013. <http://www.disabilitycompendium.org/compendium-statistics/employment/2-3-civilians-with-hearing-disabilities-ages-18-64-living-in-the-community-for-the-u-s->

5. Employment data for adult who are Deaf and hard –of hearing <http://www.pepnet.org/sites/default/files/employmentbrief_v5.pdf>

6. Sprint National Traditional Relay Service <http://www.sprintrelay.com/sprint_relay_services/sprint_national_traditional_relay_services.php#tty_vco>

7. TTY: TTY stands for Text Telephone. A TTY is a special device that lets people who are deaf, hard of hearing, or speech-impaired use the telephone to communicate, by allowing them to type messages back and forth to one another instead of talking and listening. A TTY is required at both ends of the conversation in order to communicate.<http://www.abouttty.com>

8. Deafness, Communication and Isolation in the Work Place <http://www.rit.edu/ntid/hccd/system/files/DEAFNESS%20COMMUNICATION%20AND%20ISOLATION.pdf>

9. Mirzaei, M. R., Ghorshi, S., & Mortazavi, M. (2014). Audio-visual speech recognition techniques in augmented reality environments. The Visual Computer, 30(3), 245-257.

10. Motion Savvy <Leap%20motion%20Tech>%20https://wefunder.com/motionsavvy (Leap Motion Tech)

11. Angrosino, M. (2007) Doing Ethnographic and Observational Research. London, England: Sage.

12. Gold, R. (1958) Roles in sociological field observation. Social Forces, 36(3):217-223.

13. McGrath, J. 1982. Dilemmatics: The study of research choices and dilemmas. In J. McGrath (Ed.), Judgment calls in research: 69-80. Beverly Hills: Sage Publications.

14. Lazar et. al. (2010) Research Methods In Human-Computer Interaction. West Sussex, England: Wiley & Sons Ltd.