Back to the Basics of Everything Data — Data Culture, Data Integration, and Snowflake’s Data Cloud

In the first of a two-part series, David Hrncir, regional technical expert at NTT DATA, and Ajay Bidani, digital enablement and insights manager at Powell Industries, go back to the basics on everything data.

Reaching the summit

We recently traveled to fabulous Las Vegas to attend and sponsor the incredible Snowflake Summit 2022. Set amongst the backdrop of the bustling Las Vegas Strip, the four-day event brought together some of the best minds in technology and data science.

As an Elite Snowflake partner and a longtime, passionate Snowflake enthusiast, NTT DATA is well-versed in Snowflake. We have a commitment to doing data better, together — especially when it comes to applying our knowledge of Snowflake when working with customers to find solutions to their data problems using the Snowflake Data Cloud.

Our data podcast, Hashmap on Tap, has featured several guests from Snowflake, discussions involving its capabilities, impacts on the data cloud (and more!), and what’s the result? You guessed it — this culmination of events presented the perfect opportunity for us to present a live podcast episode. We know it wouldn’t be Vegas without entertainment (and don’t forget the mimosas!)

So what was “on tap” for this session?

David Hrncir, Ajay Bidani and one of our clients took the stage. To properly celebrate the occasion, they popped the cork on a bottle of champagne to enjoy mimosas — a brunch favorite and definitely a fitting choice for an 11:00 a.m. session (if I don’t say so myself). They hosted a lively discussion on of topics and challenges in the field of data science today.

Data strategy

How would you respond if someone, right now, asked you the question, “What’s your data strategy?”

If your initial response is that of a deer in headlights, that’s not a good sign. On the other hand, if you can lay out 20 goals of your data strategy, then according to David, you’re on the right track, since one of the most critical pieces when it comes to working with clients is discovering their data strategy.

When asked about Powell’s data strategy and pushing towards modernization, Ajay places emphasis on the need to establish a foundation that consists of going back to the basics and keeping things simple.

To make this happen, they focus on keeping users engaged when it comes to data, acknowledging there is a gap when it comes to self-service and scalable collaboration for people, and turning insights into action.

Data strategy vs. data culture

So how does data strategy fit in when talking about data culture, and how do you promote, discuss or reinforce data culture within your organization? According to Ajay, “the conversation about data strategy is basically the exercise of trying to promote culture without using the word ‘culture.’” To encourage data culture, they focues on increasing awareness and talking about the data they already have, and taking the insights gained from that existing data to determine what data they may need in the future.

Data culture starts with leadership

When it comes to data culture, getting leadership involved is absolutely crucial in promoting, discussing or reinforcing the topic and necessary behaviors or actions. If leadership is involved in this process, it’s ultimately going to help in attaining the organization’s goals of a strong data culture with a data-driven mindset.

David shared the results of a recent survey, in which the number of companies referring to themselves as data-driven has gone down from 37% to 31%. Additionally, 60% of all data analytics projects fail, 79% of all data projects had too many errors to be trustworthy, and 87% of all data science projects never make it to production.”

He found these results to be particularly shocking— primarily because of the decrease from 37% to 31% of companies that described themselves as data-driven. In his opinion, this shift is a reversal of what we think of as being the standard in this industry.

Why a data-driven mindset matters

From Ajay’s perspective, without a data-driven mindset being pushed by leadership, failed projects will seemingly always continue, but they’re never going to get any higher. These projects may get worse and worse over time when it comes to the challenges you’ve had or investments that you’ve made. As a result, this can cause your systems to become suboptimal, and you may end up in a situation where the user experience starts to suffer. And then you lose your ability to do more because there’s not a lot of trust in what you’ve been doing.

Ajay explained: “When operating lean, you forget what it’s like to have people in the right roles as opposed to wearing multiple hats.” How would a situation like this affect an organization’s data culture?

It certainly says a lot about the data culture if you can’t “try and take those hats off” as David suggests. It’s important to recognize that there’s value in making it a necessity to be focused on people’s accomplishments and goals, and developing this mindset is incredibly important to help organizations change for the better.

Data integration for the win

t’s impossible to attend Snowflake Summit and not talk about data integration. Many leaders in all types of organizations are either looking into data integrations or will need data integration eventually. David states that “I don’t like to use ETL, ELT, ETLT, RELT — it’s just too complex to say all these different acronyms so I say DI [Data Integration] — it’s the easiest way to do it.”

What’s with all the acronyms?

What is Ajay’s perspective on organization ETL and how does Powell Industries look at it? How does he prefer to work with those different acronyms, and what do they mean to him and his organization?

According to Ajay, “I would be lying if I said that acronyms are not our best friend, and to be honest with you, we do it way too much — just like everyone else.” He continues by saying, “We definitely talk about (and use) the term ETL and ELT regularly. However, what’s unfortunate about it, in his opinion, is that it can cause you to fall into the state of mind that ‘this’ means ‘this’ because of how we experience it today.” So from his perspective, acronyms are helpful, but they shouldn’t be static — it’s important for them to continue to evolve over time.

What are the roles of data ingestion and data transformation?

When it comes to data integration, it covers a couple of topics, such as ingestion and transformation. At Powell, Ajay shares that they have primarily been talking about those concepts individually because their focus has been on what they’ve built so far and the exact thing they’re trying to do.

However, when you’re thinking about DI, you shouldn’t be thinking of it solely as an ELT tool. When working on projects, one of the most common requests is DI for reference architecture. David states, “We are trying to build these reference architectures and playbooks, ultimately, to have more successful future implementations.”

Reference architectures and playbooks

Can reference architecture and playbooks be used as cookie cutters that work in your organization? If you want Ajay’s honest answer, he says, “Not especially. “Reference architectures serve as a good point of reference, especially for learning, but he personally prefers to take reference architecture as “a place for interrogation.”

He continues, “It’s kind of like saying, ‘These are pieces. Why are these pieces important?’ In my organization, there’s no way that we’re going to do something because someone came in and said, ‘Hey, this is the thing to do.’” Ajay’s statement is an interesting way to think about reference architectures and let the organization put things into perspective.

The first question an organization is going to ask is, “What does it cost?” The organization needs to understand the end result and the ways that it fits into the reference architecture in order to justify what you’re trying to do. Ajay believes that the reference architecture serves as an internal way to help yourself understand what is necessary to reach the desired end result. As Ajay concisely puts it, “Well, it’s reference architecture. It doesn’t tell you everything. It tells you the kinds of things you need.”

Then, the challenge lies in trying to understand those foundational lessons better as well as how they affect what an end-user would get experience-wise. Once you have a better understanding of exactly how those play into that experience, you’ll find that building experimentation into strategy — and doing it often — is obviously the key.

Experimentation tends to bring with it a new slew of buzzwords and acronyms. While these may just be acronyms, it’s important to think about the question, “How can these meanings and uses differ within the organization?”

Can you share some of those acronyms?

• Proof of Concept (PoC) — the architecture as a whole; doing things that have never been done before to achieve your end result: a good (working) piece of software architecture
• Proof of Technology (PoT) — investigating a software vendor or a service to see if their technology is capable of performing the functions it claims to be able to do
• Proof of Value (PoV) — understanding how something works when put into practice; asking “Do I get any value from this?” to determine if its benefits (like the fact that it saves time, uses less manpower, etc.) outweigh the monetary cost
• Proof of Purchase (PoP) — the ability to meet a deadline and a cost for a project; determining whether or not the cost for a project can actually be met by iterating the process

Minimal viable product

Ajay brings up the example of a minimum viable product. His philosophy is that instead of aiming for a minimum viable product (MVP), take it and add complexity to it. Look to see what future iterations could include and what it will take to get from one step to the next. These actions can help to set yourself up for success, so you don’t let yourself get trapped within the MVP. Additionally, by involving your organization in adopting this process, it will allow increased support for getting from one step to another.

Prioritizing and simplifying data

How can a minimalist mindset be used to make this concept less complex and remove some of the moving parts? David uses the analogy of a house to break this question down. Most houses aren’t carved from a giant rock — they are formed by building small parts incrementally. In other words, you have to lay the foundation first. If you try to go from one brick to a mansion and hope it works, that is essentially a recipe for disaster.

When it comes to making things simpler with tools, it’s important to look at the amount of code that’s involved — having to learn a front-end tool that requires a lot of learning could be more complex than other available tools. In terms of cost models, Ajay says that “Less is more . . Think about how many different levers do you have to pull?” It’s easiest for him to think about this concept by quantifying the amount of data as opposed to how much it’s hit (or how much compute for how many times in a day).

By quantifying the amount of data, it can make this process easier and allow the potential for planning. He says, “How many times I hit it is a little bit more like trying to sell lean — or why I should need it less.” This conveys that it’s imperative to identify the requirements and goals of the data to ensure that business decisions are made in alignment with those factors. In David’s experience, clients that have the most success are the ones whose leadership already has a strong data mindset.

The wrap-up

What happens in Vegas stays in Vegas, so we can’t give you all the details—you had to be there to experience it! In part two of this series has David and Ajay covering a of topics on technology expansion, challenges in the modern data cloud, data warehouses and more. Check it out.