Data science was an unfamiliar space for me. To better understand the pain points of my user group, I conducted semi-structured interviews with 5 senior data scientists having an experience of 15+ years.

I was keen to understand their experience with exploratory data analysis and automated solutions.
‍

When you look at a big table with many columns, you often get lost. In the domain of big data, displaying critical information and helping people dissect information layer-by-layer: I think that's most important to me.

The feedback I recieved revolved around a lack of clear direction during analysis, and a lack of interactivity with the existing tools that leverage automation.

I was most surprised to hear a data scientist mention that they'd prefer their time consuming and outdated solutions over the tools that are present today.

I started the second round of user research with 5 usability tests that helped me evaluate how well the tool supported a data scientists' exploration workflow.

"What I find great here is that the entry is simple and does not have too much barricades. It gives people a starting point, regardless of how deep their knowledge is."

Users responded positively to the the layout of the tool and insights being presented. However, a large number of users were reluctant to trust automated insights and visualizations that they were not familiar with.

"I need to be comfortable with what you're saying is an error, I need to build some trust.

It could be possible that since I'm a data scientist I like to deal precisely with numbers and see detailed information. Otherwise I'm more suspicious."

I conducted another round of semi-structured user interviews, talking to 4 data scientists about how their trust can be built.

"Depending on the projects, you should also be able to experiment to see what behaves better. It would be very important to have that and certainly would give me confidence."

Having iterated on the prototype primarily from a usability point of view, I went back to testing with users to guage metrics for control, trust & comfort. To test these metrics, I designed tasks that were borrowed from the journey maps I had created in the previous round of research.

Having iterated on the prototype primarily from a usability point of view, I went back to testing with users to guage metrics for control, trust & comfort. To test these metrics, I designed tasks that were borrowed from the journey maps I had created in the previous round of research.

Out of the various different stages in an ML pipeline, accelerating exploratory data analysis continues to be a challenging problem space. This is largely due to the numerous ways in which a data scientist can approach a problem and choose to conduct his analysis. A lot of modern tools have tried to automate this analysis and in the process left humans out of the loop.

At the end of this project, I propose a human-driven framework, that presents automated insights dynamically as the user drills down on a dataset. The structured organization of the system allows users to ease into a dataset, and not get overwhelmed by the large number of columns. By designing overview visualizations that give users quick ideas about the dataset, the proposed framework gives direction to their exploration and provides them with various entrypoints into a dataset.

There remains more work to be done before modern automation-based tools can be introduced to the workflows of experts. Throughout my research, I’ve learned about the skepticism in userswhile relying on results from processes not designed by them. Simply being transparent about the methods being used was not enough to gain the trust of users. As evidenced by the 3-step flow I designed, systems must accommodate various methods for users to further investigate the accuracy of insights.