As data professionals, we all want to work on cool data problems and to be successful in those projects. However, what often comes as a surprise is that the definition of cool and measure of success evolves as you go from school to industry. A paradigm shift happens when we go from working on data projects in a controlled environment (like school, bootcamps, etc.) to tackling data projects in the real world. With my years of experience as a data professional, and serving as a mentor for aspiring data scientists, I would like to share a few practitioner insights from myself and my colleagues at Doximity about what the day-to-day of a data scientist is really like.
Data Is the Answer, but What Is the Question?
“Real-world data is messy, incomplete, and often we find ourselves with more questions than answers after an analysis. The good thing is data professionals like questions.” — Lilla Czako, senior data engineer
Data professionals constantly re-evaluate not only if we are solving the problem right but also, more importantly, if we are solving the right problem. In school, someone always knows the question; the problem set is clear, at least in the teacher’s mind. However, our stakeholders rarely know precisely what needs to be done. They will often come to you with either a concern or a hope and look towards you to provide data context, fill gaps, push back if required, and shape the overall problem statement. Our job is to translate vague ideas into a quantifiable problem statement that can then be translated into mathematical language.
Once you arrive at a quantifiable problem statement, it is not the end of the road either. It is quite possible that the available data does not support the kind of analysis you originally envisioned (missing values, hidden confounding variables, sparse features, sparse data points, etc.). A situation like this will further tune and refine your problem statement.
Adaptability, practicality, and being comfortable with ambiguity are the three most valuable skills in a data scientist.