Data Sharing Community

From CDQ
Jump to navigation Jump to search

Welcome to the Portal of the CDQ Data Sharing Community

What's new? (RSS)

New data source integrated: Hochschulrektorenkonferenz (DE.HRK) (17 April 2025)

Summary

We are pleased to announce the integration of a new data source into the Business Partner Lookup service: Hochschulrektorenkonferenz (DE.HRK).

Details

The Hochschulrektorenkonferenz provides official information about all German universities.

Each record may include the following details:

  • University official name
  • University short name
  • Address

Enhanced Weekly Mail Summaries Powered by AI (16 April 2025)

We’re excited to announce improvements to our weekly emails, which now include AI-generated summaries for charts based on customer data. These summaries help you quickly understand what has changed and identify areas where data quality can be enhanced.

Key Enhancements

  • Automated AI Insights: Each Monday, the email digest now features AI-generated conclusions derived from customer data, offering clear insights into changes and data quality trends.
  • Detailed Dashboard Summaries: The weekly emails provide comprehensive information, including:
    • Business Partner Distribution: Overview of partner distribution in the mirror, segmented by country.
    • Data Defects Analysis:
      • Information on the top 10 countries by record volume.
      • Analysis of critical data quality defects, focusing on the Top 5 most violated rules per country.
      • Consideration of business impact and industry sector specifics.
    • Data Quality Progress: Trends and measurements of data quality over time for the Top 10 countries, using mean values per week from the last 10 calendar weeks.
    • Access to Past Release Notes: Direct links to previous week’s release notes are provided for additional context and follow-up.


How to set up email subscription?

If you’re not yet receiving these updates, follow these simple steps:

  1. Open the Global Settings app.
  2. Select the workspace for which you wish to receive the emails.
  3. Ensure the workspace is set to the production type (click Edit Workspace and choose the appropriate type from the selector).
  4. Click Manage Notification.
  5. Create your notification configuration.
  6. Add the desired users to the recipient list and save your settings.


This enhanced automation aims to simplify your monitoring process and improve the quality of insights drawn from your data. Enjoy a more streamlined approach to keeping abreast of vital information every week!

Business Partner Status Available in the Tax Guard Identifier Qualification Results (14 April 2025)

Business Partner status is now provided together with identifier qualification results in Tax Guard. An additional response object called businessPartnerStatus indicates the Business Partner status: type, value and reference to the Wiki. Moreover, the value is present in the First Time Right application (as a result of the Qualify button execution) in case the status is different than UNKNOWN or ACTIVE. Many data sources (like VIES, AT.FON or BZST) do not provide the status of a Business Partner. Thus, in such cases we return UNKNOWN value. All the possible status values can be found here: https://meta.cdq.com/Business_partner/status/type.

... further results

Data model

An important prerequisite for collaborative data management is a common understanding of the shared data. For the CDQ Data Sharing Community, this common understanding is specified by the CDQ Data Model. The concepts of this model are defined and documented in this wiki which can be used as a business vocabulary. Moreover, the wiki provides a machine-readable interface to reuse this metadata by using semantic annotations.

This is a graph with borders and nodes that may contain hyperlinks.

Data maintenance procedures

A procedure is a common standard or "how-to" for a specific data management task. Within the CDQ Data Sharing Community, companies agree on such procedures to ensure similar rules and guidelines for similar tasks. For several countries, the CDQ Wiki provides such information, e.g. data quality rules, trusted information sources, legal forms, or tax numbers. Try

or select another country from the list.

Data sources

Active data sourcesRecords
Data source BR.RF65,168,461
Data source CDQ.INTEL53,424,836
Data source VIES50,000,000
Data source FR.RC40,588,052
Data source US-CA.BER8,743,900
Data source GB-EAW.CR8,653,270
Data source US-FL.BER6,189,994
Data source JP.CR5,607,635
... further results
The CDQ Data Sharing Community uses a collaboratively managed reference data repository. This incorporates the integration of external data sources for enriching or validating business partner and address data. Examples of available data sources are 316 countries (e.g. WORLD (World), AT (Österreich, Austria, Autriche, 奥地利), BE (Belgien, Belgium, Belgique, België, 比利时)), 993 legal forms (e.g. ), and 72 active business partner data sources (e.g. Data source CDQ.POOL, Data source VIES, Data source CH.UIDR).

Metadata and Standards: Metadata-driven Data Quality

Data quality plays a pivotal role in ensuring compliance with legal, regulatory, and industry standards. One of the core challenges in achieving high data quality is adhering to dynamic data requirements that evolve due to changes in national regulations. These requirements vary by country, making it essential for businesses to track and update compliance criteria continuously.

In many countries, official company information is available as Open Data, but the lack of a standardized data model or provision method complicates the process of integrating this data. The Data Sharing Community actively collaborates to identify global data requirements and reference data sources, whether Open Data or commercial.

Short description
Managed reference data for administrative areas with language-specific terms and short names according to ISO 3166-2.
Managed reference data for bank accounts worldwide.
Managed reference data for types of identifiers per country.
Basic data concepts of CDQ Cloud Services.
Managed reference data about compliance lists considered in the sanction and watchlist screening services
Managed reference data for countries with language-specific names and short names according to ISO 3166-2.
Documentation of data quality rules with explanation and technical constraints to validate business partner data records.
Data quality rule functions are methods implemented in a programming language for being used in data quality rule implementations. They can be e.g. used in custom data quality rules similar to functions employed by business users in popular spreadsheet applications such as Microsoft Excel.
Managed reference data for legal forms with official and commonly used abbreviations and corresponding country.
Managed reference data for localities, such as exonyms.
Managed reference data for post codes
Managed reference data for postal delivery points, such as Post Office Boxes used for identification, extraction, harmonization and standardization.
Managed reference data for issuing bodies of identifiers
Managed reference data for thoroughfares of type Street (CDQ.POOL) used for harmonization and standardization.

Data Quality Rules

Transformation of human-documented data requirements into executable data quality rules is mostly a manual IT effort. Changing requirements cause IT efforts again and again. Some checks, e.g. tax number validity (not just format!), require external services. Other checks, e.g. validity of legal forms, require managed reference data (e.g. legal forms by country, plus abbreviations). Continuous data quality assurance (i.e. batch analyses) and real-time checks in workflows often use different rule sets.

Data requirements and related reference data are collected and updated collaboratively by the Data Sharing Community. Data quality rules are derived from these requirements automatically. All data quality rules are executed behind 1 interface, in real-time. Batch jobs and single-record checks use the same rule set and can be integrated by APIs.

For proving that a data quality rule is content-wise correct we maintain supporting document(s) per data quality rule which share the rule's source. This could be:

  • a public authority source
  • any other trustful webpage
  • a data standard of a specific community member

We manage the URL (if any), a screenshot of the relevant parts (if any) and the source's name (e.g. Community member data standard, European Commission, National ....) See Identifier format invalid (SIREN (France)) as an exemplary rule that was specified and implemented based on information provided by the OECD.