Heading to the last week of the REU program, I did not expect the time to go by so fast. But regardless I have had a wonderful experience so far. I learned so much from this program from how to conduct research, gather information to develop a literature review, self-accountability, independent research, gathering data, developing a study plan, developing stimuli, giving presentations, analyzing data, and many more skills I have improved and learned. I cannot thank the program’s advisor and my team project for helping me out throughout the program.

This ninth week of the REU program was more dedicated to continuing to gather participants and conducting studies to gather data. Last week I was able to get only eight partipants to be involded and conduct the study. I was hoping to have 10 studies done by the end of last week but I was not able to do that but throughout this week I was able to figure out times to meet with partipants who were interested and I was able to gather 12 more partipants and finish with gathering data for 20 studies. In between the studies, I worked on uploading and labeling the partipants’ information in the correct folder such as their consent form by labeling it to their study ID number like this, “ConsentFormID_XX” and as well labeled the recording to their id number, “ID_XX.” Once that is completed I went ahead and transcribe their follow-up questions responses onto a google document for the purpose to delete the recording in the future and conducting thematic analysis on their responses.

This week’s main focus was to gather participants for our study. Ramzy and I discussed how we plan on balancing responsibility for conducting studies. I was willing to conduct all of the studies as a lot of the participants we gathered are ASL users and Ramzy is not proficient in ASL enough to feel confident to be a part of the study. But regardless I enjoy doing the studies. I send out our recruitment flyer on various social media platforms and informed my friends as well. When I get participants who are interested they would email me and I would send them a “when2meet” poll to figure out what time they are available to conduct the study and I also send out the demographic form to get to know them more and to make sure that they meet the basic requirements to be a part of the study. This process repeats for every time I get a new participants who is interested.

This week was another slow week as this was the week where our team has been waiting on our IRB approval for us to be able to start recruiting participants for our study and start collecting data. In between this week, we took the time revising other components that we already developed to pass the time while waiting for the IRB approval. We looked back at the methodology section of our paper to add some revisions and more content. We as well made some further tweaks and modifications to our surveys and finalized all the details to be ready for the real participant testing. Towards the end of the week, we had another pilot study with Sam Sit and this helped us finalize and make some minor corrections to the study. While we were waiting. Ramzy and I discussed how the flyer could be improved to make it feel more engaging for our audience and Ramzy went ahead and took the initiative to revise the flyer.

With this week’s goal in mind to finish up developing the experimental materials required to conduct the study. I received all the clips necessary to make the stimuli for the study. I used a video editing software called Kapwing to develop captions and make the conditioned videos as well. Kapwing has a really cool feature that allows for easy creations of captions which then can be converted into SRT files which is a form of caption file. Kapwing has this included slider where I can pinpoint which part of the video and add text for that certain caption line. So, first I went ahead and made all the default captions for each video and uploaded it to the correct location in our shared drive. This includes uploading the SRT file as well. Then for the first condition which includes Memoji within the captions, I took all the Memoji pictures that Ramzy has given me and I have cropped them to fit better within the videos. Then I went ahead and place the appropriate Memoji for each caption line. This was relativity easy as I was able to match each caption line to a Memeoji and it was able to “snap” to each other so I did not have to worry about timing. Once I was done with those videos I moved on to making the second conditioned videos which were including text descriptions within captions [HERE] which was relatively easy to do. This was done by editing the original SRT caption file. Once I was all done with making the conditioned videos I went ahead and uploaded it to the correct file location in the shared drive.

Towards the end of the fourth week. I was able to receive some video clips to develop captions and developed the conditioned captions for it as well from Ramzy. There were a lot of different obstacles that we faced as a team in this week when it comes to developing our experimental study material. These obstacles were mostly on Ramzy’s side where his internet has not been consistent or reliable enough for him to create those video clips and uploaded them. In the meantime, I continued to iterate the Methodology section of our research and left some parts unknown as we would need more information before adding. I have continued reading some more articles that Dr. Raja suggesting in our folder and start digesting them to consider where to add within our literature review as that will give more information and support to our research questions based on emotions and paralinguistic information in captions.

I entered the fourth week of the program with a lot of motivation. This week’s main focus was to develop the Study Design plan and how we want to approach our research questions in terms of a study. I’ve met with my teammate at the beginning of the week to discuss our current study plan and start listing down our initial steps to conduct our study. I showed my teammate what software I use to modify captions files such as .srt and .ass files and how to burn them on videos using the command line. We also discussed reforming our research questions. To state them again, our research questions are: the first one being; how do we incorporate paralinguistic and emotive information in captions while retaining speaker identification? And our second research question; will incorporating paralinguistic and emotive information through the use of speaker identification help improve compression and retention for deaf and hard-of-hearing?

I entered the third week with the focus of finishing up the Literature Review and cleaning up any little issues that appear in the Literature Review. While the finalization of the Literature Review was happening, the development of the Update 1 - PowerPoint discussing our findings in our Literature Review and our research questions was occurring. We wanted to explain our findings so far that will help reinforce the research questions that we want to further research into. We had our program-wide weekly meeting and each project team gave their presentation on their Literature Review findings and their research questions. As my team, which includes Ramzy and I, we presented our work so far and got tremendous feedback from our mentors. Such as that we need more visualization in our presentation. We also need to make connections to other articles/studies relating to different aspects of captions as they will have important implications for understanding captions even though our research question is more focused on speaker identification and emotions. We do have these documents recorded in our spreadsheet listing all the articles. Even though those such articles have nothing to relate to speaker identification and emotions, it is still important to discuss other research that already exists and make connections to them.

I entered the second week of the program with much more stability. I was aware of what I needed to do and what goals I need to accomplish for this week. After finding multiple articles/papers relating to Captions UI/UX and digesting the content within each article/paper. We were able to start constructing the literature review. We have made tremendous progress in terms of the literature review and shaping it to fit our research questions/goals. While constructing it and getting feedback from our mentors. We were able to get a more narrow idea of what we want to do for our research. While focusing on captions UI/UX we want to see how to combine speaker indicators and emotions and display that within the captions. One possible avenue of research could include the use of Apple’s memoji feature to show speaker indicators and emotions within captions.

My first week working in a new environment with new people all over the country was an interesting experience! Especially when it is all being done virtually as well! Before meeting everyone I was unsure of what topic I was going to do. After meeting everyone, I got assigned to work on a special type of REU project along with one other student researcher as my teammate for this project, Ramzy Oncy-Avila! Our project focuses on “captions UI/UX” and we are really excited to see what we can discover and contribute some solutions to improve the captions experience for deaf and hard of hearing people and all other people who use captions.