Building a healthy data culture and embracing bad ideas with Hilary Mason

March 9, 2021

Hilary Mason is a co-founder of Hidden Door, a social game platform for kids to discover and create stories with AI. She is a former founder & CEO of Fast Forward Labs, a machine intelligence research company acquired by Cloudera in 2017. She also holds a role as a Data Scientist in Residence at Accel Partners.

Hilary is also a blogger, speaker, author and dedicates her time to talent development as a Board member at the Anita Borg institute for women in technology and as a co-founder at hackNY, a non-profit organization raising the next generation of engineering talent in the New York community.

From enterprises to startups

At the beginning of our talk, Hilary shared her journey into the world of data science. Throughout her career, Hilary observed the rise of data science and ML as distinct practices. She evangelized for adoption of ML in enterprises as a part of her role at Fast Forward Labs and later at Cloudera, and advised companies on how they can apply ML and AI in their businesses.

What was her motivation for moving away from the enterprise world and starting Hidden Door - another company from the ground up? "I've been passionate about using tech as a tool to help users reduce creative friction and spur creativity." With Hidden Door, she's doubling down on that and building a digitally mediated game that is collaborative and based on the principles of games like Dungeons and Dragons and Roblox. Her goal is to develop an interactive social game platform for childen, designed for safety, enabled by AI and natural language processing (NLP).

Building a healthy data culture

As a founder of Fast Forward Labs, Hilary advised companies on embracing AI & ML opportunities, shaped data strategies and helped them prevent common pitfalls. We've asked her to share some of the learnings around how companies can succeed in building a healthy data culture.

"First and foremost, success requires rigorous approach to understanding the problems you're trying to solve and the ability to adapt as you learn, driven by both data and business intuition." In addition to this, Hilary mentions that it's crucial to have open-minded leadership that is adaptable and open to change based on new learnings. Lastly, she talks about the importance of availability of data across the organization and treating data as an "accepted and democratized tool".

So what are some of the characteristics of organizations with healthy data cultures? Hilary summarizes this in 4 key points.

• Leadership that either understands data well, or does not have the necessary skillset, but is humble enough to accept that and have close advisors helping them understand, prioritize and spot business opportunities.
• Organizational design. Hilary talks about the importance of how the data teams are stacked against the rest of the organization. A well-functioning data organization is one where the customer and product leaders are well-represented in data endavours and the workflows are conductive to cross-functional communication. "This sounds very simple, but I have not yet found a solution for organizational design that works for every context. You need to go by the key principles and find out what works for you."
• Start simple. Hilary's advice for companies is to strip away the complexity at the beginning and "start with the simplest subset of the problem possible. Build end-to-end pipelines, and get feedback loop from the product as soon as possible". This way, teams can avoid throwing resources at projects that will not see business application, and conversely, have a good case for introducing more complexity as they see business value being derived.
• Understanding that managing a data science team is not the same as managing a software engineering team. As opposed to engineering projects, data science is defined by exploration - you don't have a clear end goal and the scope is changing as you gather findings. Hilary also makes the case that returns are calculated differently for a data science team or a project. "Any investment into data applications progresses the full organization and is an investment into the future of data capabilities, it does not only relate to that one specific use case."

"I sincerely believe that every CEO and product leader will have to have a data-driven understanding of how their product and features are performing, but that's not the case today. The folks who do have that understanding are already ahead."

Failure modes in embracing data oppportunities

We talked about key failure modes that Hilary has encountered at her clients' organizations.

In her view, there are often two extreme mindsets counter-productive to embracing data opportunities. Firstly, it's the tools-first mindset. "Focusing on methodologies and technical tooling as the first order of business misses the point entirely", says Hilary. The other extreme is being too data-driven. "You can tweak data to push for what you want to see anyway". Leaders should course-correct for a balance between the two, relying on deep business understanding and intuitition, complementing their understanding with data insights and enacting change as a result of an interplay between the two.

Hilary also hints on the importance of organizational design for success of data strategies. "People best positioned to learn from data are often decoupled from it", she says. Isolation and silo-ing of data analytics and data science teams with product- and customer-facing teams causes a ton of great oppportunities to go unexplored. She suggests it should be the data scientists' responsibility to sit down with their business partners and help them identify and explore data opportunities, regardless of the organizational boundaries.

Lastly, we spoke about the lack of involvement of data scientists in the ideation stage of product development. Often, involving the data science team is the last thing on everyone's to-do list, that only comes up once the product has been built. "At that point, you might have missed plenty opportunities to instrument product more intelligently and possibly incorporate useful ML features", she says.

Organizing data science teams

We doubled down on the organizational design perspective and discussed how to best distribute data science capabilities so that they can support the organization effectively. There's multiple ways to approach this issue and there's no silver-bullet solution, but Hilary mentions the core principle she uses in her work: "when you sit people together, you're optimizing for those functions to collaborate at the expense of friction between them and everyone else".

The key questions that leaders should ask themselves are:

• Where are the frictions in my organization that need to be addressed?
• What functions do we need to bring closer together, given the business needs?
• What have we underappreciated historically that we need to compensate for today?

Hilary also notes it's important to approach organizational design in an iterative way - if you do make an organizational change today, you should be open to revisiting this decision again in 18 - 24 months. Revisit the key questions, understand the impacts of previous decisions and explore whether a change might be needed based on new business priorities.

These principles apply in large organizations as much as they do in early-stage companies. As a founder or an early employee at a startup, you can embrace the responsibility that you have to set the example of making data-driven decisions for anyone that will join the company after you.

"I used to think that dynamically changing organizational structure is a sign of failure, but now I see it as a sign of organizational maturity. Changing your organization means you understand changing business priorities and that you're investing into improving efficiency and productivity of your teams."

Embracing bad ideas

"I always get really nervous when I come into a company and I see that everything they're working on is an obviously good idea and not much risk of failure on the table", Hilary says. To move forward at a pace and innovate, you often need to start with bad ideas, even those that might seem silly or impossible. "If there's no risk in what you're working on, then you're likely leaving a ton of potential on the table".

Hilary surfaces bad ideas proactively, and with her teams asks them to share the worst ideas they can think up. This often leads to interesting discussions and eventually, to ideas that are high risk, high reward. The amount of risk is also her heuristic for differentiating the good from the mediocre. "Good is easy to recognize, but mediocre often looks a lot like good."

"Good ideas are often obvious, low risk but low reward. Embrace bad ideas."

What's next in ML/AI and data science?

Lastly, we asked Hilary to share some of the key trends in the data science ecosystem that she's most excited about:

• Professionalization and standardization of data science practice. Hilary talked about the evolution of data science role, and how it's very different across companies in terms of skillset needed, tooling and methodologies used, management and where data scientists sit in an organization. She predicts we'll see more convergence on the vocabulary and definition of the role.
• More standardization across feature management and metadata management. Especially in the light of deep learning evolution and its real-life applications.
• Vertical Machine learning applications. Hilary predicts the trend to continue where companies build out ML models for niche use cases, such as accounting, healthcare, fraud detection. "Vertical has definitely been the key trend for the past few years and there's still a ton of unexplored potential."
• NLP technology for language generation. NLP for language generation and vector models are areas where the pace of progress is accelerating. "I'm extremely excited to see how NLP can be applied to creativity and giving people tools for expressing themselves in an empowering way."

References mentioned on the podcast

• People to follow: Matt Brandwein, co-founder of Hidden Door, previously head of ML products at Cloudera
• Books to read: Data Driven, Creating a Data Culture by Hilary Mason and DJ Patil

Made with 💙 by the Deepnote team. Deepnote is a new kind of data science notebook - Jupyter-compatible with real-time collaboration and running in your browser. Try it out for free.

For questions, say hi on Twitter or reach out at deepindata@deepnote.com.