Why talk about CI/CD today?
In the ever-evolving world of cloud and data, certain practices that we take for granted are being profoundly challenged. CI/CD (Continuous Integration/Continuous Deployment) is undergoing a fascinating transformation. This evolution could make our current practices seem like prehistory – like when we explain to young developers that we used FTP to deploy to production.
Logical CI/CD: a revolutionary new approach
“I call it logical CI/CD,” introduces Frédéric Adet with the enthusiasm of a developer who has just discovered a major optimisation in his code. “It’s the ability to deploy and integrate environments easily at the logical level, thanks to Snowflake’s functionalities. This approach radically differs from traditional physical CI/CD, which we’re all familiar with and which populates our cloud providers and on-premise architectures. As a data architect might say: ‘Why create ten physical copies when one logical copy will suffice?’”
The economic stakes: the sinews of war
Cloud invoices sometimes feel excessive, but logical CI/CD relieves finance departments.
“The first thing I see is the possibility of making drastic savings,” explains Frédéric. These savings can be seen on several levels:
- Reduced physical integration and deployment costs
- Optimising human resources
- Significant reduction in compute bills from cloud providers
The Snowflake clone: when copying weighs nothing
At the heart of this revolution is an almost magical feature: Snowflake Zero Copy Cloning. “It’s as simple as the product itself,” explains Ilyès Mehaddi. “‘Zero Copy Cloning’ is a managed Snowflake feature that enables you to create a logical copy of a data product instantly.”
Imagine creating a perfect copy of your database instantly without physically duplicating the data.
The practical benefits of cloning :
- Instant creation of work environments (from a production environment, for example),
- Zero cost as long as there are no new entries,
- Complete environmental insulation,
- Possibility of working on real data without risk,
- Fast time to production.
An end to production start-up nightmares
“We’ve all been through production changeovers where we were all shaking in our boots,” recalls Ilyès. Logical CI/CD changes all that by allowing us to work on exact copies of the production, eliminating the traditional last-minute surprises.
The technical ecosystem: a modern, efficient stack
The ideal technical stack for setting up this new-generation CI/CD is:
Main ingredients :
- Git: the versioning orchestra conductor
- Snowflake: the heart of the reactor
- Snowflake CLI: the command line interface
- Terraform: for infrastructure as code
- Streamlit: for user interfaces
“It’s very easy to come up with simple, effective recovery or business continuity plans using these processes,” Frédéric points out.
The latest Snowflake innovations
Snowflake is constantly expanding its ecosystem with new features that will light developers’ eyes. Ilyès tells us about the latest additions:
Modern integration :
- Native connection with Git
- Integrated development environments with Notebooks
- Simple SQL functions for Git integration
- API integration for synchronising deliverables
“We’ll also be able to find functions that we’ve set up in extremely simple SQL functions so that we can create connections with git,” explains Ilyès.
Real-life use cases
Logical CI/CD opens the door to many scenarios that were previously complex or costly:
Development teams :
- Creation of on-demand test environments
- Experimentation with no impact on costs
- Tests on real data in complete safety
For data scientists :
- Isolated experimental environments
- Access to fresh copies of production data
- Rapid iterations of models
For ops teams :
- Simplified management of environments
- Reduced maintenance tasks
- Advanced deployment automation
Vision of the future
“In tomorrow’s world, we will have a logical CI/CD developed as early as possible to have a sustainable core team and a rapid time-to-market value proposition for all companies at the lowest possible cost”, prophesies Frédéric.
Conclusion: The need for change
Logical CI/CD is more than just a technical evolution – it’s a paradigm shift in how we think about developing and deploying data solutions. A famous developer once said, “The best line of code is the one you don’t need to write.” Similarly, the best environment may be the one you don’t need to create physically.
In a world where resource efficiency and optimisation are becoming crucial, logical CI/CD appears to be a natural and necessary evolution. It reminds us that sometimes the best way forward is not to run faster but to take a different path.
Editor’s note: No production environment was abused in the making of this interview.
Devoteam helps you lead the (gen)AI revolution
Partner with Devoteam to access experienced AI consultants and the best AI technologies for tailored solutions that maximise your return on investment. With over 1,000 certified AI Consultants and over 300 successful AI projects, we have the expertise to meet your unique needs.