i-spark and AI/ML

20 June 2024 | 3 minutes of reading time

Machine Learning (ML) and Artificial Intelligence (AI) have become crucial tools for businesses today. Over the years, at i-spark we have  been steadily integrating this into our workflow. We now use platforms that employ AI like Databricks, as well as create our own custom solutions in Python or R. However, using ML and AI effectively requires not only technical expertise but also a clear understanding of realistic business goals and privacy regulations.

i-spark and Machine Learning (ML) 

Members of our team have been engaging with ML since around 2003. Back then, this was simply known as predictive modeling. We focused on response models for commercial mailings and clustering models for segmentation, using tools like SPSS and SAS. Today, these processes fall under the umbrella of ML. We now leverage advanced tools such as Databricks and Dataiku, or we develop custom solutions in Python or R. Cloud computing has significantly expanded our capabilities. However, defining the right business cases and managing expectations remains challenging. While ML is highly valuable, it is not a magic solution.

i-spark and Artificial intelligence (AI)

We make use of AI-based tools to streamline our work processes. Some examples are UseMotion for intelligent agenda management, and ChatGPT as a sounding board for refining ideas. These tools have become indispensable to us, and we cannot imagine working without them. We believe every company should adopt such tools to enhance their efficiency and productivity. 

When using AI (tools), several key considerations come into play:

Code generation

We use AI for writing code, but primarily for simple syntax tasks. For example, when dealing with multiple SQL variants (Snowflake, GBQ, Redshift, T-SQL, etc.), we might get a little confused about date formats and we use AI for the correct date syntax. This is practical and efficient; the key is knowing what you want to achieve. For more complex data transformations, we still rely on our expertise, as ChatGPT and similar tools have not yet reached the required proficiency level.

Data analysis

We ensure client data remains secure and is not used for training models, therefore we do not provide client data to arbitrary AI tools. However, we do use tools with built-in AI capabilities, like PowerMetrics or GoodData, if they are part of our client's architecture. Alternatively, we create conditions that allow safe AI utilization.

Creating Environments and Analytics Stacks

We set up controlled environments where clients can use the LLM API’s from the GPT powered OpenAI API, Gemini by Google, DBRX by Databricks or Arctic by Snowflake.. This allows clients to choose the best option for their needs and ensures a secure and efficient workflow.

Conclusion

The line between ML and AI can be blurry. (See ‘AI, more than a buzzword?’ for further breakdown of AI vs. ML.) Although we generally agree that what is currently termed AI does not fully represent what artificial intelligence is supposed to be, we will align with the commonly used terminology for practical purposes. 

As we look towards the future, the next critical step for companies is to prioritize data quality. High-quality data is the foundation upon which effective AI solutions are built. Without clean, accurate, and well-structured data, even the most advanced AI systems will fail to deliver meaningful insights. Integrating ML and AI into business processes is an ongoing effort, but with the right approach and expertise, the potential benefits are substantial. Our aim is to support businesses in leveraging these technologies for sustained growth and innovation.