Big Data profile picture

Unifying metadata governance across Amazon SageMaker and Collibra Amazon Web Services (AWS) and Collibra have built a new integrated solution that demonstrates the integration between the Collibra Platform and the next generation of Amazon SageMaker. In this post, we take a closer look at the integration, describe the use cases it enables, walk through the architecture, and show how to implement the solution in your environment.
https://aws.amazon.com/blogs/b....ig-data/unifying-met

image

Discover the world at Altruu, The Discovery Engine
    Big Data profile picture

Orchestrate data processing jobs, querybooks, and notebooks using visual workflow experience in Amazon SageMaker Today, we are excited to launch a new visual workflows builder in SageMaker Unified Studio. With the new visual workflow experience, you don’t need to code the Python DAGs manually. Instead, you can visually define the orchestration workflow in SageMaker Unified Studio, and the visual definition is automatically converted to a Python DAG definition that is supported in Airflow.This post demonstrates the new visual workflow experience in SageMaker Unified Studio.
https://aws.amazon.com/blogs/b....ig-data/orchestrate-

image

Discover the world at Altruu, The Discovery Engine
    Big Data profile picture

Introducing Jobs in Amazon SageMaker This post demonstrates how the new jobs experience works in SageMaker Unified Studio.
https://aws.amazon.com/blogs/b....ig-data/introducing-

image

Discover the world at Altruu, The Discovery Engine
    Big Data profile picture

Compaction support for Avro and ORC file formats in Apache Iceberg tables in Amazon S3 In this post, we explore how Amazon S3 Tables has expanded its automatic compaction capabilities to include Avro and ORC file formats for Apache Iceberg tables, alongside the previously supported Parquet format. Through performance testing with over 20 billion events, the capability demonstrates significant query performance improvements ranging from 12% to 40% when using compacted tables compared to non-compacted tables across different file formats.
https://aws.amazon.com/blogs/b....ig-data/compaction-s

image

Discover the world at Altruu, The Discovery Engine
    Big Data profile picture

Revenue NSW modernises analytics with AWS, enabling unified and scalable data management, processing, and access Revenue NSW, Australia's principal revenue management agency, successfully modernized its analytics infrastructure using AWS services. In this blog post, we show how the organization transformed its on-premises data environment into a unified, scalable cloud-based solution using Amazon Redshift, AWS Database Migration Service, Amazon AppFlow, and AWS Glue.
https://aws.amazon.com/blogs/b....ig-data/revenue-nsw-

image

Discover the world at Altruu, The Discovery Engine