We’re excited to announce the launch of Projects, a new organizational enhancement to the Coalesce data transformation platform.
Projects are logical groupings of work in Coalesce that help customers organize their data warehousing efforts by a specific functional mission. Much like folders in a shared Google drive, every Project contains its own universe of work.
Each Project comes with its own set of user permissions and is tied to its own git repository, set of development Workspaces, and deployable Environments. Data experts can take advantage of particular user-defined nodes (UDNs) and macros that are attached to a given Project.
Why we’re building Projects
This launch is an important first step in enabling data teams from different departments to gain visibility into data projects across their organization while transforming their data on a single platform. We’ve worked with a number of organizations that struggle with siloed data projects, which are the result of departmental data marts. It’s an issue we keep encountering and Coalesce’s data lineage capabilities put us in a unique position to help solve it.
Typically, different departments have little to no visibility into data initiatives that are happening across their company. To get the data they need for their own initiatives, teams have a few options– either seek out data directly from source systems and rely on central IT to get that data ready for consumption, or use data from other departments without a full understanding of its origins, quality, or how it has been manipulated over time. As a result, data teams are stuck using data sets that are less than ideal for their project, which in turn often leads to massive rework efforts.
When you take into account that businesses are constantly changing their rules and requirements for how data is used, it becomes almost impossible for those changes to be reflected consistently in analyses produced by different data teams.
Current State: Data preparation and analysis is isolated in silos as a result of departmental data warehousing initiatives.
How Coalesce users can best benefit from Projects
With Projects, different data teams can work together with equal footing on the same transformation platform. This particular feature helps people manage team-based data initiatives more effectively, by focusing work on a specific view of their data warehouse that’s relevant to their particular initiative. It also allows users to govern and monitor access to Projects and support better security best practices by only provisioning access to team members that need it. Coalesce customers can use Projects to better understand the impact of changes made to their data with deeper observability into data lineage, quality, and the current state of their data warehouse.
The future of Projects
Projects moves us closer to our vision for the Coalesce platform: to make data transformation more collaborative and give data teams the power to meaningfully connect with and use the work of other teams. Coalesce already enables data professionals to do this by using standardized data patterns to automate manual transformations, but we’re driving towards a future where every data team has full observability across data projects occurring within their company, and can leverage that work with confidence in its quality. Our built-in column-level lineage capabilities are helpful in this regard, but we’re focused on expanding them by threading metadata through previously siloed data projects. This cross-project linking will be key to reflecting the flow of data changes across the work of different data teams, surfacing essential context to data users, and bringing clarity to entire organizations on how data is defined, used, and evolves over time.
It’s an exciting time to be on the cutting edge of data transformations and these capabilities will have an incredible impact on our customers, so be sure to watch this space.
With Projects in Coalesce
- Cross-Project linking provides column-level lineage from raw data sources to different projects.
- The output of one Project can be used as input to another Project, with full contextual background on data lineage.
- Teams have confidence in the origins and evolution of how the data has been. transformed as it moves through different pipelines.
Projects is currently available in private preview – learn more in our documentation and contact your sales representative to join the release.
