Difference between revisions of "Data Sharing Community"

From CDQ
Jump to navigation Jump to search
 
(Undo revision 715161 by Kaihuener (talk))
Tag: Undo
 
(349 intermediate revisions by 5 users not shown)
Line 1: Line 1:
__NOTOC__
+
__INDEX__
{|
+
<div id="mainpage">
|style="width:60%;vertical-align:top;padding-right:20px"|
+
{| style="border-collapse:collapse; width:100%;"
== Corporate Data League (CDL) ==
+
 
Business partner data management is heavily redundant: Many companies manage data for the same entities such as country names and codes, bill-to, ship-to, and ordering addresses, or legal hierarchies of customers and suppliers. For this reason, the [http://www.cdq.ch CC CDQ] is setting up a trusted network of user companies for sharing such data: The '''Corporate Data League''' (CDL). [https://www.corporate-data-league.ch Get a brief overview...]
+
| colspan="2" style="padding:0;" |
 +
<div class="mainpage-top">
 +
<h2> Welcome to the Portal of the CDQ Data Sharing Community</h2>
 +
<div class="content" style="text-align:center;">
 +
The CDQ Data Sharing Community is a [[Data Sharing Community Members|trusted network of user companies]] to manage [[CDQ Data Model|business partner data]] [[Capability/data sharing|collaboratively]].
 +
 
 +
<div class="portals" style="text-align:center;">
 +
{{main page topic | icon = fa-map-o | page = Data model | label = Data model}}&nbsp;&nbsp;&nbsp;&nbsp;
 +
{{main page topic | icon = fa-life-ring | page = Support | label = Support}}
 +
 
 +
[[Metadata and Standards]]&nbsp;· [[Metadata and Standards/data quality rules|Data Quality Rules]]&nbsp;· [[Reference data|Data Sources]] &nbsp;· [[Data maintenance procedures|Procedures]]
 +
</div></div></div>
 +
 
 +
|-
 +
| style="padding:0; vertical-align:top; width:50%;" |
 +
<div class="mainpage-left">
 +
<h2> [[Release note|What's new?]] ({{#ask:
 +
[[Category:Release note]]
 +
 
 +
|mainlabel = Release note
 +
|?has headline = News
 +
|?has description = Description
 +
|?has release date = Release date
 +
 
 +
|sort=Has release date
 +
|order=descending
 +
|format = feed
 +
|type = rss
 +
|title = Release notes RSS
 +
|limit=100
 +
}}) </h2>
 +
<div class="content">{{#ask:
 +
[[Category:Release note]]
 +
 
 +
|mainlabel = Release note
 +
|?has headline = News
 +
|?has description = Description
 +
|?has release date = Release date
 +
|?has technical key
  
In this {{SITENAME}}, we provide information about
+
  |sort=Has release date
* '''Data model''': Structures, definitions, business rules. [[Data model|Read more...]]
+
  |order=descending
* '''Web services''': Documentation and examples. [[Web services|Read more...]]
+
  |format = template
* '''Reference data''': Lists of e.g. {{#ask: [[Category:Country]] | format=count }} countries and {{#ask: [[Category:Legal form]] | format=count }} legal forms. [[Reference data|Read more...]]
+
  |template=News list
* '''Web application''': Web application to evaluate CDL functionality. [[Web application|Read more...]]
+
  |limit=3
*'''Getting integration started''': Technical integration guideline. [[Getting integration started|Read more...]]
+
}}<!-- --></div></div>
|style="vertical-align:top;"|
 
== News ==
 
{{#ask:[[Category:News item]]
 
| ?has date =
 
| ?has summary =
 
  | sort = has date
 
  | order = desc
 
  | format = template
 
  | template = news list
 
  | limit = 3
 
| searchlabel=
 
  
 +
<div class="mainpage-left">
 +
<h2>[[Data source/CDQ/data model|Data model]]</h2>
 +
<div class="content">
 +
{{#show: Data source/CDQ/data model | ? has description}}
 +
{{#ask:
 +
[[Category:data model concept]] [[Is part of::Data source/CDQ]] [[Has concept role::ROOT]] OR [[Category:data model concept]] [[Is part of::Data source/CDQ]] [[Has concept role::NODE]] [[Has parent concept.Has concept role::ROOT]]
 +
| ?Has parent concept
 +
| format = graph
 +
| nodelabel = displaytitle
 +
| graphsize = 6
 +
| graphname = datamodel
 
}}
 
}}
{{#ask:[[Category:News item]]
+
<!-- | graphlink = yes -->
| ?has date =
+
</div></div>
| ?has summary =
 
| sort = has date
 
| order = desc
 
  | limit = 0
 
| searchlabel=''View older news &hellip;''
 
}}
 
  
News reader integration: {{#ask:[[Category:News item]]
+
<div class="mainpage-left">
| sort = has date
+
<h2> [[Data maintenance procedures]] </h2>
| order = desc
+
<div class="content">
| format = feed
+
{{#show: Data maintenance procedures | ? has description}} Try
| searchlabel = RSS
+
* {{#queryformlink:form=Country procedure|link text=CH ({{#show: Address/country/CH | ?Has monolingual name.Text }})|link type=link|query string=Country procedure[country]=Address/country/CH}},
| title = Corporate Data League –- News
+
* {{#queryformlink:form=Country procedure|link text=DE ({{#show: Address/country/DE | ?Has monolingual name.Text }})|link type=link|query string=Country procedure[country]=Address/country/DE}},
| description = Latest news from the Corporate Data League
+
* {{#queryformlink:form=Country procedure|link text=PL ({{#show: Address/country/PL | ?Has monolingual name.Text }})|link type=link|query string=Country procedure[country]=Address/country/PL}},
}} · {{#ask:[[Category:News item]]
+
* {{#queryformlink:form=Country procedure|link text=US ({{#show: Address/country/US | ?Has monolingual name.Text }})|link type=link|query string=Country procedure[country]=Address/country/US}},
| sort = has date
 
| order = desc
 
| format = feed
 
| type = atom
 
| page = full
 
| searchlabel = Atom
 
| title = Corporate Data League -– News
 
| description = Latest news from the Corporate Data League
 
}}
 
|}
 
  
== CTI Funding ==
+
or select another country from the list.
[[File:LogoCTI.png|thumb|300px|link=https://www.kti.admin.ch/kti/en/home.html|Commission for Technology and Innovation]] The Corporate Data League (CDL) was started from a research and development project that was funded by the [https://www.kti.admin.ch/kti/en/home.html Commission for Technology and Innovation (CTI)] in 2014. In another running research project, the CDL team elaborates new ways for data validation with [[business rules]] together with researches from the [http://www.iwi.unisg.ch Institute of Information Management] at the [http://www.unisg.ch University of St. Gallen]. Also these activities are funded by the CTI until June 2016.
+
</div></div>
  
== Data Model ==
+
| style="padding:0; vertical-align:top; width:50%;" |
<!--[[File:Data model digraph CDLModel dot.png|right|thumb|300px]]-->
 
A common understanding of shared data is a mandatory prerequisite for cross-corporate data management. For the Corporate Data League, the [[data model|CDL data model]] specifies this common understanding. Currently, the following {{#ask: [[Category:data model attribute]] | format=count }} [[:Category:data model attribute|attributes]] are documented. [[Web services|Read more...]]
 
{{#ask: [[Category:data model attribute]]
 
| mainlabel = Concepts
 
| ?Is part of = Entity
 
| ?Has definition = Definition
 
| limit = 5
 
}}
 
  
== Business Rules ==
+
<div class="mainpage-right">
To ensure data quality, {{#ask: [[Category:business rule]] | format=count }} [[business rules]] are defined. The following rules are currently implemented using semantic web technology:
+
<h2> [[Data sources]] </h2>
{{#ask: [[Category:business rule]]
+
<div class="content">
  | mainlabel = Business Rule
+
{|
  | ?Has formalized representation=Definition
+
| style="vertical-align:top; width:300px; padding: 0;" |{{#ask:[[Category:Data source]] [[Release status::7||6||5||4]] [[Has number of records::+]]  
  | limit = 5
+
  | mainlabel = Active data sources
 +
  | ?Has number of records = Records
 +
| sort = Has number of records
 +
| order = desc
 +
  | limit=8
 
}}
 
}}
 +
| style="vertical-align:top; padding-top: 8px;" |{{#show: reference data | ? has description}}
 +
|}
 +
</div></div>
  
== Web Services ==
+
<div class="mainpage-right">
From an integration perspective, CDL [[web services]] are the most important component of the CDL infrastructure. They provide the technical link between your business applications and the CDL cloud services. We follow the [[wikipedia:REST|REST]] design principle for web services which allows for lightweight interface design and easy integration. Of course, all web services are also available at [[wikipedia:WSDL|WSDL]] interfaces. Currently, we provide the following {{#ask: [[Category:Web service]] | format=count}} [[web services]]. [[Web services|Read more...]]
+
<h2> [[Metadata and Standards|Metadata and Standards: Metadata-driven Data Quality]] </h2>
{{#ask: [[Category:Web service]]
+
<div class="content">
  | mainlabel=Web service
+
{{#show: Metadata and Standards | ?has short description }}
  | ? Web service category=Category
+
{{#ask:[[Category:Metadata package]]  
| ? Has description = Description
+
  | mainlabel = Metadata
| sort=Web service category
+
  |?has short description = Short description
| limit=5
 
 
}}
 
}}
 +
</div></div>
  
== Reference Data ==
+
<div class="mainpage-right">
The Corporate Data League uses a collaboratively managed [[reference data|reference data repository]] for data validation. Currently,
+
<h2> [[Metadata and Standards/data quality rules]] </h2>
* {{#ask: [[Category:Country]] | format=count }} countries (e.g. {{#ask: [[Category:Country]] | format=list | limit=10 | order=random | searchlabel= }}) and
+
<div class="content">
* {{#ask: [[Category:Legal form]] | format=count }} legal forms (e.g. {{#ask: [[Category:Legal form]] | format=list | limit=10 | order=random | searchlabel= }})
+
{{Map/data quality rules}}
are maintained. [[Reference data|Read more...]]
+
{{#show: Data quality rules | ?has description }}
 +
|}
 +
</div></div>
 +
</div>__NOTOC__ __NOEDITSECTION__
  
== Web Application ==
+
__INDEX__
[[File:DemoApplication.png|right|thumb|400px]]
 
The [[web application|CDL web application]] provides access to all functionality of the CDL cloud infrastructure such as
 
* [[Web application for data cleansing|data cleansing]],
 
* [[Web application for duplicate analysis|duplicate analysis]] and
 
* [[Web application for banking data fraud management|banking data fraud management]].
 
'''Enjoy testing, and please [[Contact|let us know]] about your feedback!'''
 

Latest revision as of 12:29, 18 May 2025

Welcome to the Portal of the CDQ Data Sharing Community

What's new? (RSS)

Simplified Data Transformation After Mapping Changes (8 July 2025)

You no longer need to use the API to transform data when there's a change in assigned mapping. Now, you can perform data transformation directly in the APPs interface.


To apply transformations:

  1. Navigate to Data Mirror Management.
  2. Select your data source.
  3. Click on Start Transformation.
  4. Choose the country scope (optional).
    • If no country is selected, the system will transform data for all BPs in the mirror where the mapping change is relevant.


This update streamlines your workflow and reduces dependency on API calls.

Monitor Status for Business Partners (27 June 2025)

We’re excited to introduce a new Status feature in the Monitors module of the Data Clinic app. This enhancement provides visibility into the current state of Business Partners (BPs) within Augmentation and Data Quality Profiling monitors, helping users make informed decisions when downloading or reviewing BP data.

Key Benefits:

  • Avoid consuming incomplete data by easily identifying BPs that are mid-reprocessing.
  • Monitor BP readiness and take action only when data is up to date.
  • Streamline workflows by accessing real-time status data directly in the UI or via API.


Status Overview:

Each BP in a monitor is now assigned one of four statuses:

  • Ready – The BP has up-to-date results and is ready for download.
  • In Progress – The BP is currently undergoing processing or reprocessing.
  • Retry – The BP requires processing has failed and needs to be reprocessed.
  • Blocked – The BP cannot be processed due to an issue that needs to be resolved.


Where to See It:

  • Configuration Page: Get an overview of how many BPs are in each status category across your monitor.
  • Data Review (Single Mode): Check the individual status of a selected BP during manual review.
  • API Access: Status information is also available programmatically via our API responses.


This feature aims to improve data accuracy, transparency, and efficiency when working with monitored BPs.

Augmented Business Partner Report & Legal Entity Report v3 (26 June 2025)

We’re excited to announce two enhancements to our reporting suite: the Augmented Business Partner Report and an updated Legal Entity Report.

Augmented Business Partner Report

This new report delivers the full results of augmentation monitoring for your business partners. You’ll see every proposed value alongside its original “before” data, plus all Update Assessment details—action taken (added, modified, deleted), classification, data provenance, similarity scores, and modification timestamp. Just like the existing Update Report, you can filter by summary classification and creation date to zero in on exactly what matters.

Legal Entity Report (v3)

The revamped Legal Entity Report leverages our latest curation logic to surface official register data—legal status, VAT registration, and any local attributes—alongside the same augmentation monitoring insights (action, classification, provenance, scores, and modification date). Both the new and legacy versions remain available, giving you flexibility to evaluate the improvements and switch over at your own pace.

... further results

Data model

An important prerequisite for collaborative data management is a common understanding of the shared data. For the CDQ Data Sharing Community, this common understanding is specified by the CDQ Data Model. The concepts of this model are defined and documented in this wiki which can be used as a business vocabulary. Moreover, the wiki provides a machine-readable interface to reuse this metadata by using semantic annotations.

This is a graph with borders and nodes that may contain hyperlinks.

Data maintenance procedures

A procedure is a common standard or "how-to" for a specific data management task. Within the CDQ Data Sharing Community, companies agree on such procedures to ensure similar rules and guidelines for similar tasks. For several countries, the CDQ Wiki provides such information, e.g. data quality rules, trusted information sources, legal forms, or tax numbers. Try

or select another country from the list.

Data sources

Active data sourcesRecords
Data source BR.RF66,349,272
Data source CDQ.INTEL53,424,836
Data source VIES50,000,000
Data source FR.RC41,793,664
Data source GB-EAW.CR8,868,373
Data source US-CA.BER8,842,285
Data source US-FL.BER6,350,007
Data source JP.CR5,646,543
... further results
The CDQ Data Sharing Community uses a collaboratively managed reference data repository. This incorporates the integration of external data sources for enriching or validating business partner and address data. Examples of available data sources are 316 countries (e.g. WORLD (World), AT (Österreich, Austria, Autriche, 奥地利), BE (Belgien, Belgium, Belgique, België, 比利时)), 993 legal forms (e.g. ), and 72 active business partner data sources (e.g. Data source CDQ.POOL, Data source VIES, Data source CH.UIDR).

Metadata and Standards: Metadata-driven Data Quality

Data quality plays a pivotal role in ensuring compliance with legal, regulatory, and industry standards. One of the core challenges in achieving high data quality is adhering to dynamic data requirements that evolve due to changes in national regulations. These requirements vary by country, making it essential for businesses to track and update compliance criteria continuously.

In many countries, official company information is available as Open Data, but the lack of a standardized data model or provision method complicates the process of integrating this data. The Data Sharing Community actively collaborates to identify global data requirements and reference data sources, whether Open Data or commercial.

Short description
Managed reference data for administrative areas with language-specific terms and short names according to ISO 3166-2.
Managed reference data for bank accounts worldwide.
Managed reference data for types of identifiers per country.
Basic data concepts of CDQ Cloud Services.
Managed reference data about compliance lists considered in the sanction and watchlist screening services
Managed reference data for countries with language-specific names and short names according to ISO 3166-2.
Documentation of data quality rules with explanation and technical constraints to validate business partner data records.
Data quality rule functions are methods implemented in a programming language for being used in data quality rule implementations. They can be e.g. used in custom data quality rules similar to functions employed by business users in popular spreadsheet applications such as Microsoft Excel.
Managed reference data for legal forms with official and commonly used abbreviations and corresponding country.
Managed reference data for localities, such as exonyms.
Managed reference data for post codes
Managed reference data for postal delivery points, such as Post Office Boxes used for identification, extraction, harmonization and standardization.
Managed reference data for issuing bodies of identifiers
Managed reference data for thoroughfares of type Street (CDQ.POOL) used for harmonization and standardization.

Data Quality Rules

Transformation of human-documented data requirements into executable data quality rules is mostly a manual IT effort. Changing requirements cause IT efforts again and again. Some checks, e.g. tax number validity (not just format!), require external services. Other checks, e.g. validity of legal forms, require managed reference data (e.g. legal forms by country, plus abbreviations). Continuous data quality assurance (i.e. batch analyses) and real-time checks in workflows often use different rule sets.

Data requirements and related reference data are collected and updated collaboratively by the Data Sharing Community. Data quality rules are derived from these requirements automatically. All data quality rules are executed behind 1 interface, in real-time. Batch jobs and single-record checks use the same rule set and can be integrated by APIs.

For proving that a data quality rule is content-wise correct we maintain supporting document(s) per data quality rule which share the rule's source. This could be:

  • a public authority source
  • any other trustful webpage
  • a data standard of a specific community member

We manage the URL (if any), a screenshot of the relevant parts (if any) and the source's name (e.g. Community member data standard, European Commission, National ....) See Identifier format invalid (SIREN (France)) as an exemplary rule that was specified and implemented based on information provided by the OECD.