What’s new with Google Data Cloud

July 28 - August 1

• AlloyDB Omni now supports Kubernetes Operator 1.5.0 and PostgreSQL ver. 16.8.0/15.12.0: We have launched AlloyDB Omni Operator 1.5.0 and database versions 16.8.0/15.12.0. This major release delivers a significant step forward in enterprise readiness, including support for OpenShift operations, high availability/disaster recovery, and critical operational improvements like low-downtime upgrades and backups from standby.

July 21 - July 25

• Introducing partitioned index for BigQuery vector search: When creating a vector index on a partitioned BigQuery table, you now have the option to also partition your vector index. Partitioning your vector index significantly reduces query costs and improves search accuracy for vector searches that utilize pre-filtering on the partitioning column.By partitioning your vector index, BigQuery can apply partition pruning to both your table and your vector index when you use a filter on the partitioning column in your vector search. This means BigQuery only scans the relevant partitions, decreasing I/O costs. Additionally, pre-filtering on the partitioning column makes your vector searches less likely to miss relevant results. This feature is particularly beneficial if most of your vector searches target specific partitions using pre-filters. You can only partition TreeAH vector indexes, and the PARTITION BY clause used for the vector index must match the one used for the original table. Read more about the partitioned indexes in vector search.
• Datastream now supports BigLake Iceberg tables in BigQuery: Customers can now easily replicate data from different supported sources (MySQL, Postgres, SQLserver, Oracle,Salesforce and MongoDB ) of Datastream into BigLake Managed Tables for use cases spanning across open lakehouse, Enterprise grade storage for analytics, streaming and AI. Streaming to BigLake Iceberg tables lets you store data in a cost-effective way in the PARQUET format. By doing this, you can keep your data in a Cloud Storage bucket while using BigQuery for querying and analysis.

• Cloud SQL Write Endpoint for Advanced DR: Cloud SQL is excited to announce the GA of Write Endpoint to make Advanced Disaster Recovery (DR) seamless for customers (Documentation). This feature enhances application resilience during instance failovers and switchovers, ensuring customer applications remain connected to the primary instance without manual intervention.The write endpoint is now available in GA for MySQL and PostgreSQL instances of Enterprise Plus Edition. It already exists for SQL Server instances. 

• Vertical Scaling for Memorystore for Valkey and Memorystore for Redis Cluster: Using Vertical Scaling, Memorystore customers can now effortlessly scale their Memorystore nodes up or down ensuring optimal cluster sizing for varying workloads. Previously, node types were immutable post-deployment, hence customers only had the option for horizontal scaling (in and out) changing the number of shards in the cluster. 

• Database Migration Service (DMS) supports migrations from SQL Server to AlloyDB for PostgreSQL in GA: Customers can now use DMS to migrate their databases from SQL Server to AlloyDB for PostgreSQL . This migration offers seamless experience, which offers a comprehensive SQL Server modernization framework with:
  • Automatic database schema and code conversion
  • Gemini augmented database code conversion
  • Gemini assisted PostgreSQL training and code improvements
  • Low-downtime, CDC based data movement

July 14 - July 18

• Trust and security are central to Conversational Analytics. Designed to gain the benefits of Google’s most capable AI models, Conversational Analytics offers a powerful and insightful natural language experience that is secure and trustworthy, meaning you can realize the full potential of generative AI with confidence, while keeping your data under control. Learn more here.
• Turn questions into queries with the Conversational Analytics API. The Conversational Analytics API, now in preview, integrates multiple AI-powered tools to process user requests, including Natural Language to Query (NL2Query) and a Python code interpreter for generating responses, simplifying data science. Learn more here.
• Introducing BigQuery Soft Failover: Greater Control Over Disaster Recovery. BigQuery now offers "soft failover," giving administrators options over failover procedures. Unlike "hard failover" for unplanned outages, soft failover minimizes data loss for planned activities like disaster recovery drills or workload migrations. It initiates failover only after all data is replicated to the secondary region, guaranteeing data integrity. This feature is available via BigQuery UI, DDL, and CLI, providing enterprise-grade control for disaster recovery, confident simulations, and compliance without risking data. Learn more here. 

July 7 - July 11

• [Webinar] Join us for a session on "Build Smart Apps with Ease: Gen AI, Cloud SQL, and Observability for Faster Development." This webinar dives deep into mastering the essentials of building powerful Gen AI applications using Google Cloud technologies. Discover the complete Gen AI application development lifecycle, get a live demonstration of the new Application Design Center (ADC) for rapid app deployment, and explore its seamless integrations with frameworks like LangChain, LlamaIndex, and LangGraph. Plus, learn about the new MCP Toolbox for Databases to enhance the manageability and security of your GenAI agents, and understand critical operational considerations, including Cloud SQL Enterprise Plus features for performance, scalability, high availability, and disaster recovery.

June 23 - June 27

• Looker developers gain speed and accuracy with debut of Continuous Integration. Continuous Integration for Looker helps streamline code development workflows, boost the end-user experience, and gives developers the confidence to deploy changes faster. Learn more here.

• Code Interpreter brings advanced data science capabilities to Conversational Analytics. Code Interpreter helps answer complicated questions, tapping into Python to perform advanced analysis on your Looker data. Learn more here.

June 16 - June 20

• Standardize your business terminology with Dataplex business glossary. Want to standardize business terminologies and build a shared understanding across the enterprise? Dataplex business glossary is now GA within Dataplex Universal Catalog, providing a central, trusted vocabulary for your data assets, streamlining data discovery, and reducing ambiguity — leading to more accurate analysis, better governance, and faster insights. Learn more here.

• Looker Core on Google Cloud is now FedRAMP High authorized.  The need to protect highly sensitive government data is a top priority. Looker Core on Google Cloud enables users to explore and chat with their data via AI agents using natural language, and create dashboards and self-service reports. Learn more here.

• Fast Dev Mode Transition Speeds Looker Developers. A new Labs feature, Fast Dev Mode Transition, improves the performance of Development Mode on your Looker instance by loading LookML projects in read-only mode until a developer clicks the Create Developer Copy button for the project. Learn more here.
• Datastream now supports MongoDB as a Source (in Public Preview): You can now easily replicate data from MongoDB source into BigQuery and  Cloud Storage  for advanced analytics, reporting, and to power generative AI applications. Datastream offers MongoDB connectivity for both Replica Sets and Sharded Clusters. This includes support for self-managed MongoDB deployments as well as the fully managed AtlasDB service.
•  Private Service Connect (PSC) on existing Cloud SQL instances (GA): Cloud SQL now offers the ability to enable Private Service Connect (PSC) on existing instances that currently utilize Private Service Access (PSA). This new functionality, generally available for PostgreSQL, MySQL, and SQL Server engines, eliminates the previous requirement of creating new instances for PSC adoption. Customers can now transition their existing PSA instances to PSC without data migration. 
• Cloud SQL for SQL Server - E+ Recommender: The Enterprise Plus recommender helps customers identify SQL Server instances that would benefit from an upgrade to the Cloud SQL Enterprise Plus Edition. It offers insights into current performance metrics, and emphasizes how Enterprise Plus features (such as the data cache and memory-optimized machines) can boost performance. Additionally, the recommender includes a convenient button for direct navigation to the instance settings page, enabling users to perform the upgrade easily. 
• AlloyDB - PSC Service Automation: With this launch, AlloyDB significantly improves the connectivity configuration experience for Private Service Connect (PSC), by automatically creating PSC endpoints in the customer VPC and exposing the IP address of the endpoint directly through the AlloyDB API, enabling seamless PSC adoption at scale.

June 9 - June 13

• Introducing Pub/Sub Single Message Transforms (SMTs), to make it easy to perform simple data transformations such as validate, filter, enrich, and alter individual messages as they move in real time right within Pub/Sub. The first SMT is available now: JavaScript User-Defined Functions (UDFs), which allows you to perform simple, lightweight modifications to message attributes and/or the data directly within Pub/Sub via snippets of JavaScript code. Learn more in the launch blog. 
• Serverless Spark is now generally available directly within BigQuery. Formerly Dataproc Serverless, the fully managed Google Cloud Serverless for Apache Spark helps to reduce TCO, provides strong performance with the new Lightning Engine, integrates and leverages AI, and is enterprise-ready. And by bringing Apache Spark directly into BigQuery, you can now develop, run and deploy Spark code interactively in BigQuery Studio. Read all about it here. 
• Next-Gen data pipelines: Airflow 3 arrives on Google Cloud Composer: Google is the first hyperscaler to provide selected customers with access to Apache Airflow 3, integrated into our fully managed Cloud Composer 3 service. This is a significant step forward, allowing data teams to explore the next generation of workflow orchestration within a robust Google Cloud environment. Airflow 3 introduces powerful capabilities, including DAG versioning for enhanced auditability, scheduler-managed backfills for simpler historical data reprocessing, a modern React-based UI for more efficient operations, and many more features.

June 2 - June 6

• Enhancing BigQuery workload management: BigQuery workload management provides comprehensive control mechanisms to optimize workloads and resource allocation, preventing performance issues and resource contention, especially in high-volume environments. To make it even more useful, we announced several updates to BigQuery workload management around reservation fairness, predictability, flexibility and “securability,” new reservation labels, as well as autoscaler improvements. Get all the details here. 
• Bigtable Spark connector is now GA: The latest version of the Bigtable Spark connector opens up a world of possibilities for Bigtable and Apache Spark applications, not least of which is additional support for Bigtable and Apache Iceberg, the open table format for large analytical datasets. Learn how to use the Bigtable Spark connector to interact with data stored in Bigtable from Apache Spark, and delve into powerful use cases that leverage Apache Iceberg in this post.
• BigQuery gets transactional: Over the years, we’ve added several capabilities to BigQuery to bring near-real-time, transactional-style operations directly into your data warehouse, so you can handle common data management tasks more efficiently from within the BigQuery ecosystem. In this blog post, you can learn about three of them: efficient fine-grained DML mutations; change history support for updates and deletes; and real-time updates with DML over streaming data.
• Google Cloud databases integrate with MCP: We announced capabilities in MCP Toolbox for Databases (Toolbox) to make it easier to connect databases to AI assistants in your IDE. MCP Toolbox supports BigQuery, AlloyDB (including AlloyDB Omni), Cloud SQL for MySQL, Cloud SQL for PostgreSQL, Cloud SQL for SQL Server, Spanner, self-managed open-source databases including PostgreSQL, MySQL and SQLLite, as well as databases from other growing list of vendors including Neo4j, Dgraph, and more. Get all the details here.