Ratings and Reviews 0 Ratings

Total
ease
features
design
support

This software has no reviews. Be the first to write a review.

Write a Review

Ratings and Reviews 0 Ratings

Total
ease
features
design
support

This software has no reviews. Be the first to write a review.

Write a Review

Alternatives to Consider

  • Parasoft Reviews & Ratings
    152 Ratings
    Company Website
  • Google Cloud Platform Reviews & Ratings
    61,066 Ratings
    Company Website
  • TIMi Reviews & Ratings
    68 Ratings
    Company Website
  • Gearset Reviews & Ratings
    305 Ratings
    Company Website
  • DropTrack Reviews & Ratings
    191 Ratings
    Company Website
  • pCloud Business Reviews & Ratings
    189 Ratings
    Company Website
  • Virtuoso QA Reviews & Ratings
    131 Ratings
    Company Website
  • RunMyJobs by Redwood Reviews & Ratings
    424 Ratings
    Company Website
  • Flagsmith Reviews & Ratings
    43 Ratings
    Company Website
  • Stonebranch Reviews & Ratings
    183 Ratings
    Company Website

What is Deequ?

Deequ is a groundbreaking library designed to enhance Apache Spark by enabling "unit tests for data," which helps evaluate the quality of large datasets. User feedback and contributions are highly encouraged as we strive to improve the library. The operation of Deequ requires Java 8, and it is crucial to recognize that version 2.x of Deequ is only compatible with Spark 3.1, creating a dependency between the two. Users of older Spark versions should opt for Deequ 1.x, which is available in the legacy-spark-3.0 branch. Moreover, we also provide legacy releases that support Apache Spark versions from 2.2.x to 3.0.x. The Spark versions 2.2.x and 2.3.x utilize Scala 2.11, while the 2.4.x, 3.0.x, and 3.1.x releases rely on Scala 2.12. Deequ's main objective is to conduct "unit-testing" on data to pinpoint possible issues at an early stage, thereby ensuring that mistakes are rectified before the data is utilized by consuming systems or machine learning algorithms. In the upcoming sections, we will illustrate a straightforward example that showcases the essential features of our library, emphasizing its user-friendly nature and its role in preserving data quality. This example will also reveal how Deequ can simplify the process of maintaining high standards in data management.

What is Apache Spark?

Apache Spark™ is a powerful analytics platform crafted for large-scale data processing endeavors. It excels in both batch and streaming tasks by employing an advanced Directed Acyclic Graph (DAG) scheduler, a highly effective query optimizer, and a streamlined physical execution engine. With more than 80 high-level operators at its disposal, Spark greatly facilitates the creation of parallel applications. Users can engage with the framework through a variety of shells, including Scala, Python, R, and SQL. Spark also boasts a rich ecosystem of libraries—such as SQL and DataFrames, MLlib for machine learning, GraphX for graph analysis, and Spark Streaming for processing real-time data—which can be effortlessly woven together in a single application. This platform's versatility allows it to operate across different environments, including Hadoop, Apache Mesos, Kubernetes, standalone systems, or cloud platforms. Additionally, it can interface with numerous data sources, granting access to information stored in HDFS, Alluxio, Apache Cassandra, Apache HBase, Apache Hive, and many other systems, thereby offering the flexibility to accommodate a wide range of data processing requirements. Such a comprehensive array of functionalities makes Spark a vital resource for both data engineers and analysts, who rely on it for efficient data management and analysis. The combination of its capabilities ensures that users can tackle complex data challenges with greater ease and speed.

Media

Media

Integrations Supported

Amazon SageMaker Data Wrangler
Apache Mahout
Apache Zeppelin
Baidu Palo
E2E Cloud
HPE Ezmeral
HStreamDB
Jovian
Kestra
Mage Platform
OPAQUE
Okera
Onehouse
Pavilion HyperOS
Protegrity
Retina
Riak TS
Unity Catalog
ZenML
emma

API Availability

API Availability

Pricing Information

Pricing not provided

Pricing Information

Pricing not provided
Free Version

Supported Platforms

SaaS

Supported Platforms

SaaS

Customer Service / Support

Web-Based Support

Customer Service / Support

Not specified

Training Options

Documentation Hub

Training Options

Documentation Hub

Company Facts

Organization Name

Deequ

Company Website

github.com/awslabs/deequ

Company Facts

Organization Name

Apache Software Foundation

Date Founded

1999

Company Location

United States

Company Website

spark.apache.org

Categories and Features

Unit Testing

Not specified

Categories and Features

Big Data

Not specified

Data Analysis

Not specified

Data Modeling

Not specified

Query Engines

Not specified

Streaming Analytics

Data Enrichment
Data Wrangling / Data Prep
Multiple Data Source Support
Process Automation

Popular Alternatives

Spark Streaming Reviews & Ratings

Spark Streaming

Apache Software Foundation

Popular Alternatives

Apache Spark Reviews & Ratings

Apache Spark

Apache Software Foundation
dbt Reviews & Ratings

dbt

dbt Labs
MLlib Reviews & Ratings

MLlib

Apache Software Foundation
MLlib Reviews & Ratings

MLlib

Apache Software Foundation
Apache Mahout Reviews & Ratings

Apache Mahout

Apache Software Foundation