irek/anonymizer
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
anonymizer/data/HUB_DCR_output.md

2130 lines
134 KiB
Markdown
Raw Blame History

This file contains invisible Unicode characters

This file contains invisible Unicode characters that are indistinguishable to humans but may be processed differently by a computer. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

# DCR flows
**Page ID:** 415205424
**Page Link:** /display/GMDM/DCR+flows
**Overview**
------------
DCR (Data Change Request) process helps to improve existing data in source systems. Proposal for change is being created by source systems a as DCR object (sometimes also called VR - Validation Request) which is usually being routed by MDM HUB to DS (Data Stewards) either in Reltio or in Third party validators (OneKey, Veeva OpenData). Response is provided twofold:
* response for specific DCR - metadata
* profile data update as a direct effect of a DCR processing - payload
**General DCR process flow**
----------------------------
![](https://documents.lucid.app/documents/4143a333-574e-4a47-b305-ce450202ce6a/pages/VU8pD1qBoGAI?a=34871&x=-4338&y=433&w=1310&h=240&store=1&accept=image%2F*&auth=LCA%20a6c0720f3536fd0b1ad0c11f58fd86ba083ce8c39cc61b165538405a6e63dce5-ts%3D1712731701)
**High level solution architecture for DCR flow**
-------------------------------------------------
![](/download/attachments/415205424/image-2025-1-9_14-21-12.png?version=1&modificationDate=1736428872573&api=v2)
Source: [Lucid](https://lucid.app/lucidchart/4143a333-574e-4a47-b305-ce450202ce6a/edit?viewport_loc=-4371%2C-495%2C5437%2C2750%2CVU8pD1qBoGAI&invitationId=inv_5f6a4de0-884a-4859-a13b-e9db321eb4b6)
**Solution for OneKey (OK)**
----------------------------
**![](/download/attachments/415205424/image-2025-1-9_14-18-12.png?version=1&modificationDate=1736428692330&api=v2)**
**Solution for Veeva OpenData (VOD)**
-------------------------------------
**![](/download/attachments/415205424/image-2025-1-9_14-18-45.png?version=1&modificationDate=1736428725567&api=v2)**
### **Architecture highlights**
* **Actors involved**: PforceRX, Reltio, HUB, OneKey
* **Key components**: DCR Service 2 (second version) for AMER, EMEA, APAC, US tenants
* **Process details**:
+ DCRs are created directly by PforceRx using DCR's HUB API
+ PforceRx checks for DCR status updates every 24h → finds out which DCRs has been updated (since last check 24h ago) and the pulls details from each one with /dcr/\_status
+ Integration with OneKey is realized by APIs - DCRs are created with /vr/submit and their status is verified every 8h with /vr/trace
+ Data profile updates (payload) are being delivered via CSV and S3 and ETLed (VOD batch) to Reltio with Deloitte's help
+ DCRRegistry & DCRRegistryVeeva collections are used in Mongo for tracking purposes
### **Architecture highlights**
* **Actors involved**: Data Stewards in Reltio, HUB, Veeva OpenData (VOD)
* **Key components**: DCR Service 2 (second version) for AMER, EMEA, APAC, US tenants
* **Process details**:
+ DCRs are created by Data Stewards (DSRs) in Reltio via Suggest / Send to 3rd Party Validation - input for DSRs is being provided by reports from PforceRx
+ Communication with Veeva via S3<>SFTP and synchronization GMTF jobs. DCRs are sent and received in batches every 24h
+ DCRs metadata is being exchanged via multiple CSV files ZIPed
+ Data profile updates (payload) are being delivered via CSV and S3 and ETLed (VOD batch) to Reltio with Deloitte's help
+ DCRRegistry & DCRRegistryONEKEY collections are used in Mongofor tracking purposes
**Solution for IQVIA Highlander (HL)**
--------------------------------------
**![](/download/attachments/415205424/image-2025-1-9_14-20-15.png?version=1&modificationDate=1736428815707&api=v2)**
**Solution for OneKey on GBLUS - sources ICEU, Engage, GRV**
------------------------------------------------------------
### **Architecture highlights**
* **Actors involved**: Veeva on behalf of PforceRX, Reltio, HUB, IQVIA wrapper
* **Key components**: DCR Service (first version) for GBLUS tenant
* **Process details**:
+ DCRs are created by sending CSV requests by Veeva - based on information acquired from PforceRx
+ Integration HUB <> Veeva → via files and S3<>SFTP. HUB confirms DCR creation by returning file reports back to Veeva
+ Integration HUB <> IQVIA wrapper → via files and S3
+ HUB is responsible for translation of Veeva DCR CSV format to IQVIA CSV wrapper which then creates DCR in Reltio
+ Data Stewards approve or reject the DCRs in Reltio which updates data profiles accordingly.
+ PforceRx receives update about changes in Reltio
+ DCRRequest collection is used in Mongo for tracking purposes
### **Architecture highlights (draft)**
* **Actors involved**: HUB, IQVIA wrapper
* **Key components**: DCR Service (first version) for GBLUS tenant
* **Process details**:
+ POST events from sources are captured - some of them are translated to direct DCRs, some of them are gathered and then pushed via flat files to be transformed into DCRs to OneKey
---
# DCR generation process (China DCR)
**Page ID:** 164470008
**Page Link:** /pages/viewpage.action?pageId=164470008
The gateway supports following DCR types:
* NewHCP – created when new HCP is registered in Reltio and requires external validation
* NewHCOL1 – created when HCO Level 1 not found in Reltio
* NewHCOL2 – created when HCO Level 2 not found in Reltio
* MultiAffil – created when a profile has multiple affiliations
DCR generation processes are handled in two steps:
1. During HCP modification – if initial activation criteria are met, then a DCR request is generated and published to KAFKA *<env>-gw-dcr-requests* topic.
2. In the next step, the internal Camel route *DCRServiceRoute* reads requests generated from the topic and processes as follows:
1. checks if the time specified by delayPrcInSeconds elapsed since request generation – it makes sure that Reltio batch match process has finished and newly inserted profiles merge with the existing ones.
2. checks if an entity, that caused DCR generation, still exists;
3. checks full activation criteria (table below) on the latest state of the target entity, if criteria are not met then the request is closed
4. creates DCR in Reltio
5. updates external info
6. creates PfizerDataChangeRequest entity in Reltio for tracking and exporting purposes.
3. Created DCRs are exported by the Informatica ETL process managed by IQIVIA
4. DCR applying process (reject/approve actions) are executed through MDM HUB DCR response API executed by the external app manged by MDE team.
The table below presents DCR activation criteria handled by system.
| | | | | |
| --- | --- | --- | --- | --- |
| **Table 9.** DCR activation criteria | | | | |
| **Rule** | **NewHCP** | **MultiAffiliation** | **NewHCOL2** | **NewHCOL1** |
| Country in | CN | CN | CN | CN |
| Source in | GRV | GRV, MDE, FACE, EVR, CN3RDPARTY | GRV, FACE, CN3RDPARTY | GRV, FACE, CN3RDPARTY |
| ValidationStatus in | pending, partial-validated *or, if merged:* OV: notvalidated, GRV nonOV: pending/partial-validated | validated, pending | validated, pending | validated, pending |
| SpeakerStatus in | enabled, null | enabled, null | enabled, null | enabled, null |
| Workplaces count | | >1 | | |
| Hospital found | true | true | false | true |
| Department found | true | true | | false |
| Similar DCR created in the past | false | false | false | false |
### Update: December 2021
* NewHCP DCR is now created if *ValidationStatus* is *pending* or *partial-validated*
* NewHCP DCR is also created if OV ValidationStatus is *notvalidated*, but most-recently updated GRV crosswalk provides non-ov *ValidationStatus as pending* or *partial-validated* - in case HCP gets merged into another entity upon creation/modification:
* DCR request processing history is now available in Kibana via Transaction Log - dashboard API Calls, transaction type "*CreateDCRRoute*"
* DCR response processing history (DCR approve/reject flow) is now available in Kibana via Transaction Log - dashboard API Calls, transaction type "*DCRResponse*"
![](/download/attachments/164470008/China%20DCR%20-%20China%20New%20HCP%20DCR.png?version=1&modificationDate=1650871240070&api=v2)
---
# HL DCR [Decommissioned April 2025]
**Page ID:** 164470085
**Page Link:** /pages/viewpage.action?pageId=164470085
Contacts
--------
| Vendor | Contact |
| --- | --- |
| PforceRX | [DL-PForceRx-SUPPORT@pfizer.com](mailto:DL-PForceRx-SUPPORT@pfizer.com) |
| IQVIA (DCR Wrapper) | [Pfizer-MDM-Support@iqvia.com](mailto:Pfizer-MDM-Support@iqvia.com) |
As a part of Highlander project, the DCR processing flow was created which realizes following scenarios:
1. Update HCP account details i.e. specialty, address, name (different sources of elements),
2. Add new HCP account with primary affiliation to an existing organization,
3. Add new HCP account with a new business account,
4. Update HCP and add affiliation to a new HCO,
5. Update HCP account details and remove existing details i.e. birth date, national id, …,
6. Update HCP account and add new non primary affiliation to an existing organization,
7. Update HCP account and add new primary affiliation to an existing organization,
8. Update HCP account inactivate primary affiliation. Person account has more than 1 affiliation,
9. Update HCP account inactivate non primary affiliation. Person account has more than 1 affiliation,
10. Inactivate HCP account,
11. Update HCP and add a private address,
12. Update HCP and update existing private address,
13. Update HCP and inactivate a private address,
14. Update HCO details i.e. address, name (different sources of elements),
15. Add new HCO account,
16. Update HCO and remove details,
17. Inactivate HCO account,
18. Update HCO address,
19. Update HCO and add new address,
20. Update HCO and inactivate address,
21. Update HCP's existing affiliation.
Above cases has been aggregated into six generic types in internal HUB model:
1. NEW\_HCP\_GENERIC - represents cases when the new HCP object is created with or without affiliation to HCO,
2. UPDATE\_HCP\_GENERIC - aggregates cases when the existing HCP object is changed,
3. DELETE\_HCP\_GENERIC - represents the case when HCP is deactivating,
4. NEW\_HCO\_GENERIC - aggregates scenarios when new HCO object is created with or without affiliations to parent HCO,
5. UPDATE\_HCO\_GENERIC - represents cases when existing HCO object is changing,
6. DELETE\_HCO\_GENERIC - represents the case when HCO is deactivating.
General Process Overview
------------------------
**Process steps:**
1. Veeva uploads DCR request file to FTP location,
2. PforceRx Channel component downloads the DCR request file,
3. PforceRx Channel validates and maps each DCR requests to internal model,
4. PforceRx Channel sends the request to DCR Service,
5. DCR Service process the request: validating, enriching and mapping to Iqvia DCR Wrapper,
6. PforceRx Channel prepares the report file containing technical status of DCR processing - at this time, report will contain only requests which don't pass the validation,
7. Scheduled process in DCR Service, prepares the Wrapper requests file and uploads this to S3 location.
8. DCR Wrapper processes the file: creating DCRs in Reltio or rejecting the request due to errors. After that the response file is published to s3 location,
9. DCR Service downloads the response and updates DCRs status,
10. Scheduled process in PforceRx Channel gets DCR requests and prepares next technical report - at this time the report has technical status which comes from DCR Wrappper,
11. DCRs that was created by DCR Wrapper are reviewed by Data Stewards. DCR can be accepted or rejected,
12. After accepting or rejecting DCR, Reltio publishes the message about this event,
13. DCR Service consumes the message and updates DCR status,
14. PforceRx Channel gets DCR data to prepare a response file. The response file contains the final status of DCRs processing in Reltio.
Veeva DCR request file specification
------------------------------------
The specification is available at following location:
[https://pfizer-my.sharepoint.com/:x:/r/personal/chinj2\_pfizer\_com/Documents/Mig%20In-Prog/Highlander/PMO/09%20Integration/LATAM%20Reltio%20DCR/DCR\_Reltio\_T144\_Field\_Mapping\_Reltio.xlsx](https://pfizer-my.sharepoint.com/:x:/r/personal/chinj2_pfizer_com/Documents/Mig%20In-Prog/Highlander/PMO/09%20Integration/LATAM%20Reltio%20DCR/DCR_Reltio_T144_Field_Mapping_Reltio.xlsx?d=w1d677439228156d73b970b42c940bf9a&csf=1&e=ng2sEu)
DCR Wrapper request file specification
--------------------------------------
The specification is available at following link:
[https://pfizer.sharepoint.com/:x:/r/sites/HLDCR/Shared%20Documents/ReltioCloudMDM\_LATAM\_Highlander\_DCR\_DID\_PFIZER\_\_DEVMapping\_v2.1.xlsx](https://pfizer.sharepoint.com/:x:/r/sites/HLDCR/Shared%20Documents/ReltioCloudMDM_LATAM_Highlander_DCR_DID_PFIZER__DEVMapping_v2.1.xlsx?d=w7d9c08a5607e45549127034e31b16e8b&csf=1&e=4DZLxt)
---
# OK DCR flows (GBLUS)
**Page ID:** 164469877
**Page Link:** /pages/viewpage.action?pageId=164469877
Description
===========
The process is responsible for creating DCRs in Reltio and starting Change Requests Workflow for singleton entities created in Reltio. During this process, the communication to IQVIA OneKey VR API is established.  SubmitVR operation is executed to create a new Validation Request. The TraceVR operation is executed to check the status of the VR in OneKey. All DCRs are saved in the dedicated collection in HUB Mongo DB, required to gather metadata and trace the changes for each DCR request. Some changes can be suggested by the DS using "Suggest" operation in Reltio and "Send to Third Party Validation" button, the process "Data Steward OK Validation Request" is processing these changes and sends them to the OneKey service.
The process is divided into 4 sections:
1. [Submit Validation Request](/display/GMDM/Submit+Validation+Request)
2. [Trace Validation Request](/display/GMDM/Trace+Validation+Request)
3. [Data Steward Response](/display/GMDM/Data+Steward+Response)
4. [Data Steward OK Validation Request](/display/GMDM/Data+Steward+OK+Validation+Request)
The below diagram presents an overview of the entire process. Detailed descriptions are available in the separated subpages.
Flow diagram
============
![](/download/attachments/164469877/OK%20DCR%20Process%20-%20Overview%283%29.png?version=1&modificationDate=1623323894757&api=v2)
Model diagram
=============
![](/download/attachments/164469877/image-2023-11-16_16-25-55.png?version=1&modificationDate=1700148355367&api=v2)
Steps
=====
* SubmitVR
+ The process of submitting VR is triggered by the Reltio events. The process aggregates events in a time window and once the window is closed the processing is started.
+ During SubmitVR process checks are executed, getMatches operation in Relto is invoked to verify potential matches for the singleton entities.
+ Once all checks are correct new submitVR request is created in OneKey and DCR is saved in Reltio and in Mongo Cache.
* TraceVR
+ The process of tracing VR is triggered each <T> hours on Mongo DCR cache collection.
+ For each DCR the traceVR operation is executed in OneKey to verify the current status for the specific validation request.
+ Once the checks are correct the DCR is updated in Reltio and in Mongo Cache.
* Data Steward Response
+ The process is responsible for gathering changes on Change Requests objects from Reltio, the process is only accepting events without the ThirdPartyValidation flag
+ Based on the received change invoked by the Data Steward DCR is updated in Reltio and in Mongo Cache
* Data Steward OK Validation Request
+ The process is responsible for processing changes on Change Requests objects from Reltio, the process is only accepting events with the ThirdPartyValidation flag. This event is generated after DS clicks the "Send to Third Party Validation" button in Reltio.
+ The DS is "Suggesting" changes on the specified profile, these changes are next sent to HUB with the DCR event. The changes are not visible in Retlio, it is just a container that keeps the changes.
+ HUB is retrieving the "Preview" state from Reltio and calculating the changes that will send to OneKey WebService using submitVR operation
+ After successful submitVR response HUB is closing/rejecting the existing DCR in Reltio. The \_reject operation has to be invoked on the current DCR in Reltio because the changes should no be applied to the profile. Changes are now validating in the OneKey system, and appropriate steps will be taken in the next phase (export changed data to Reltio or reject suggestion).
Triggers
========
Described in the separated sub-pages for each process.
Dependent components
====================
Described in the separated sub-pages for each process.
---
# Data Steward OK Validation Request
**Page ID:** 172306908
**Page Link:** /display/GMDM/Data+Steward+OK+Validation+Request
Description
===========
The process the DS suggested changes based on the Change Request events received from Reltio(publishing) that are marked with the `ThirdPartyValidation` flag. The "suggested" changes are retrieved using the "preview" method and send to IQVIA OneKey or Veeva OpenData for validation. After successful `submitVR` response HUB is closing/rejecting the existing DCR in Reltio and additionally creates a new DCR object with relation to the entity in Reltio for tracking and status purposes. 
**Because of the ONEKEY interface limitation, removal of attributes is send to IQVIA as a comment.**
Flow diagram
============
![](/download/attachments/172306908/OK%20DCR%20Process%20-%20DS%20OK%20Validation%20Request%283%29.png?version=1&modificationDate=1630576165643&api=v2)
Steps
=====
* Event publisher publishes full enriched events to `$env-internal-[onekeyvr|thirdparty]-ds-requests-in`: DCR\_CHANGED("CHANGE\_REQUEST\_CHANGED") and DCR\_CREATED("CHANGE\_REQUEST\_CREATED")
* Only events with ExternalInfo and ThirdPartyValidation flag set to true and the Change Requests status equal to AWAITING\_REVIEW are accepted in this process, otherwise, the event is rejected and processing ends.
* HUB DCR Cache is verified if any ReltioDCR requests exist and are not in a FAILED status, then processing goes to the next step.
* DCR request that contains targetChangeRequest is enriched with the current Entity data using HUB Cache
* Veeva specific: The entity is checked, If no VOD crosswalk exists, then "golden profile" parameters should be used with below logic
* The entity is checked, If active [ONEKEY|VOD] crosswalk exists the following steps are executed:
+ The suggested state of the entity is retrieved from Reltio using the `getEntityWithChangeRequests` operation (parameters - entityUri and the changeRequestId from the DCR event).
+ Current Entity and Preview Entity are compared using the following rules: (full attributes that are part of comparing process are described **[here](#DataStewardOKValidationRequest-ONEKEYcomparator)**)
- Simple attributes (like FirstName/LastName):
* Values are compared using the equals method. 
+ if differences are found the suggested value is taken.
+ If no differences are found
- for mandatory, the current value is taken
- for optional, the none value is taken (null)
- Complex attributes (like Specialties/Addresses):
* Whole nested attributes are matched using Reltio "uri" attributes key.
* If there is a new Specialty/Address, the new suggested nested attribute is taken
+ Veeva specific: If there is a new Specialty/Addresses/Phone/Email/Medical degree\*/HCP Focus area\*, the new suggested nested attribute is taken. Since Veeva uses flat structure for these attributes, we need to calculate specialty attribute number (like specialty\_5\_\_v) to use when sending request. Attribute number = count of existing attributes +1.
* If there is no new Specialty/Address and there is a change in the existing attribute, the suggested nested change is taken. If there are multiple suggested changes, the one with the highest Rank is taken.
* If there are no changes
+ for mandatory, the current nested attribute that is connected with the ONEKEY crosswalk is taken.
+ for optional, the none nested attribute is taken (no need to send)
- Contact Affiliations / OtherHCOtoHCOAffiliation:
* If there are no changes, return current list
* If there is new Contact Affiliation with ONEKEY crosswalk, add it to current list
- Additional checks:
* If there are changes associated with the other source (different than the [ONEKEY|VOD]), then these changes are ignored and the VR is saved in Reltio with comment listing what attributes were ignored e.g.: "Attributes: [YoB: 1956], [Email: [engagetest123@test.com](mailto:engagetest123@test.com)] ignored due to update on non-[onekey|VOD] attribute."
* If attribute associated with [ONEKY|VOD] source is removed, a comment specifying what should be removed on [ONEKY|VOD] side is generated and sent to [ONEKY|VOD], e.g.: "Please remove attributes: [Address: 10648 Savannah Plantation Ct, 32832, Orlando, United States]."
+ `DCRRequest` object is created in Mongo for the flow state recording and generation of the new unique DCR ID for validation requests and data tracing.
- | DCR cache attributes | Values for IQVIA | Values for OK | Values for Veeva (R1) |
| --- | --- | --- | --- |
| type | OK\_VR | PFORCERX\_DCR | RELTIO\_SUGGEST |
| status | DCRRequestStatusDetails (DCRRequestStatus.NEW, currentDate) | | |
| createdBy | onekey-dcr-service | User which creates DCR via Suggest button in Reltio | User which creates DCR via Suggest button in Reltio |
| date | now | | |
| SendTo3PartyValidation | true (flag that indicates the DCR objects created by this process) | | |
+ Calculated changes are mapped to the OneKey `submitVR` Request and it's submitted using API REST method POST /vr/submit.
- Veeva specific:  submitting DCR request to Veeva requires creation of ZIPed CSV files with agreed structure and placed on S3 bucket
- If the submission is successful then:
- `DCRRequest.status` is updated to `SENT`with [OK|VOD] request and response details
* **DCR**entity is created in Reltio and the relation between the processed entity and the DCR entity
+ Reltio source name (crosswalk.type): *DCR*
+ Reltio relation type: HCPtoDCR or HCOtoDCR (depending on the object type)
+ DCR entity attributes:
| DCR entity attributes | Mapping for OneKey | Mapping for Veeva |
| --- | --- | --- |
| DCRID | OK VR Reqeust Id (cegedimRequestEid) | ID assigned by MDM HUB |
| EntityURI | the processed entity URI | |
| VRStatus | "OPEN" | |
| VRStatusDetail | "SENT" | |
| Comments | optionally comments | |
| SentDate | current time | |
| SendTo3PartyValidation | true | |
- Otherwise (FAILED)
- `DCRRequest.status` is updated to `FAILED`with OK request and exception response details
- **DCR**entity is created in Reltio and the relation between the processed entity and the DCR entity
* Reltio source name (crosswalk.type): `DCR`
* Reltio relation type: `HCPtoDCR`or `HCOtoDCR`(depending on the object type)
* DCR entity attributes:
* | DCR entity attributes | Mapping |
| --- | --- |
| DCRID | OK VR Reqeust Id (cegedimRequestEid) |
| EntityURI | the processed entity URI |
| VRStatus | "CLOSED" |
| VRStatusDetail | "FAILED" |
| Comments | ``` ONEKEY service failed [exception details] ``` |
| SentDate | current time |
| SendTo3PartyValidation | true |
+ The current DCR object in Reltio is closed using the \_reject operation - POST - `/changeRequests/<id>/_reject`
* Otherwise, If ONEKEY crosswalk does not exist, or the ONEKEY crosswalk is soft-deleted, or entity is EndDated: the following steps are executed:
+ DCRRequest object is created in Mongo for the flow state recording and generation of the new unique DCR ID for validation requests and data tracing.
- | DCR cache attributes | values |
| --- | --- |
| type | DCRType.OK\_VR |
| status | DCRRequestStatusDetails (DCRRequestStatus.NEW, currentDate) |
| created by | onekey-dcr-service |
| date | now |
| SendTo3PartyValidation | *true (flag that indicates the DCR objects created by this process)* |
+ *DCRRequest.status* is updated to *FAILED*and comment "No OK crosswalk available"
+ ***DCR***entity is created in Reltio and the relation between the processed entity and the DCR entity
- Reltio source name (crosswalk.type): *DCR*
- Reltio relation type: *HCPtoDCR*or *HCOtoDCR*(depending on the object type)
- DCR entity attributes:
- | DCR entity attributes | Mapping |
| --- | --- |
| DCRID | OK VR Reqeust Id (cegedimRequestEid) |
| EntityURI | the processed entity URI |
| VRStatus | "*CLOSED"* |
| VRStatusDetail | "*REJECTED*" |
| Comments | ``` No ONEKEY crosswalk available ``` |
| CreatedBy | *MDM HUB* |
| SentDate | current time |
| SendTo3PartyValidation | true |
+ The current DCR object in Reltio is closed using the \_reject operation - POST - /changeRequests/<id>/\_reject
* END
ONEKEY Comparator (suggested changes)
-------------------------------------
**HCP**
| Reltio Attribute | ONEKEY attribute | mandatory type | attribute type |
| --- | --- | --- | --- |
| ``` FirstName ``` | individual.firstName | optional | simple value |
| LastName | individual.lastName | mandatory | simple value |
| Country | ``` isoCod2 ``` | mandatory | simple value |
| Gender | individual.genderCode | optional | simple lookup |
| Prefix | individual.prefixNameCode | optional | simple lookup |
| Title | individual.titleCode | optional | simple lookup |
| MiddleName | individual.middleName | optional | simple value |
| YoB | individual.birthYear | optional | simple value |
| Dob | individual.birthDay | optional | simple value |
| TypeCode | individual.typeCode | optional | simple lookup |
| PreferredLanguage | individual.languageEid | optional | simple value |
| ``` WebsiteURL ``` | individual.website | optional | simple value |
| Identifier value 1 | individial.externalId1 | optional | simple value |
| Identifier value 2 | individial.externalId2 | optional | simple value |
| ``` Addresses[] ``` | address.country address.city address.addressLine1 address.addressLine2 address.Zip5 | mandatory | complex (nested) |
| ``` Specialities[] ``` | individual.speciality1 / 2 / 3 | optional | complex (nested) |
| ``` Phone[] ``` | individual.phone | optional | complex (nested) |
| ``` Email[] ``` | individual.email | optional | complex (nested) |
| ``` Contact Affiliations[] ``` | workplace.usualName workplace.officialName workplace.workplaceEid | optional | Contact Affiliation |
| ONEKEY crosswalk | ``` individual.individualEid ``` | mandatory | ID |
**HCO**
| Reltio Attribute | ONEKEY attribute | mandatory type | attribute type |
| --- | --- | --- | --- |
| ``` Name ``` | workplace.usualName workplace.officialName | optional | simple value |
| Country | ``` isoCod2 ``` | mandatory | simple value |
| ``` OtherNames.Name ``` | workplace.usualName2 | optional | complex (nested) |
| TypeCode | workplace.typeCode | optional | simple lookup |
| ``` WebisteWebsiteURL ``` | workplace.website | optional | complex (nested) |
| ``` Addresses[] ``` | address.country address.city address.addressLine1 address.addressLine2 address.Zip5 | mandatory | complex (nested) |
| ``` Specialities[] ``` | workplace.speciality1 / 2 / 3 | optional | complex (nested) |
| ``` Phone[] (!FAX) ``` | workplace.telephone | optional | complex (nested) |
| ``` Phone[] (FAX) ``` | workplace.fax | optional | complex (nested) |
| ``` Email[] ``` | workplace.email | optional | complex (nested) |
| ONEKEY crosswalk | ``` workplace.workplaceEid ``` | mandatory | ID |
Triggers
========
| Trigger action | Component | Action | Default time |
| --- | --- | --- | --- |
| **IN** Events incoming | mdm-onekey-dcr-service:ChangeRequestStream | process publisher full change request events in the stream that contain ThirdPartyValidation flag | realtime: events stream processing |
Dependent components
====================
| Component | Usage |
| --- | --- |
| [OK DCR Service](/display/GMDM/OK+DCR+Service) | Main component with flow implementation |
| [Veeva DCR Service](/display/GMDM/Veeva+DCR+Service) | Main component with flow implementation |
| [Publisher](/display/GMDM/Publisher) | Events publisher generates incoming events |
| [Hub Store](/display/GMDM/Hub+Store) | DCR and Entities Cache |
---
# Data Steward Response
**Page ID:** 164469841
**Page Link:** /display/GMDM/Data+Steward+Response
Description
===========
The process updates the DCR's based on the Change Request events received from Reltio(publishing). Based on the Data Steward decision the *state* attribute contains relevant information to update DCR status.
Flow diagram
============
![](/download/attachments/164469841/OK%20DCR%20Process%20-%20Update%20DS%20response%282%29.png?version=1&modificationDate=1623325846840&api=v2)
Steps
=====
* Event publisher publishes simple events to `$env-internal-[onekeyvr|veeva]-change-requests-in`: DCR\_CHANGED("CHANGE\_REQUEST\_CHANGED") and DCR\_REMOVED("CHANGE\_REQUEST\_REMOVED")
* Only the events without the ThirdPartyValidation flag are accepted, otherwise, the event is Rejected and the process is ended.
* Events are processed in the Stream and based on the *targetChangeRequest.state* attribute decision is made
+ If the state is APPLIED or REJECTS, DCR is retrieved from the cache based on the *changeRequestURI*
- If DCR exists in Cache The status in Reltio is updated
| DCR entity attributes | Mapping |
| --- | --- |
| VRStatus | *CLOSED* |
| VRStatusDetail | state: APPLIED → ACCEPTED state: REJECTED → REJECTED |
- Otherwise, the events are rejected and the transaction is ended
+ Otherwise, the events are rejected and the transition is ended.
Triggers
========
| Trigger action | Component | Action | Default time |
| --- | --- | --- | --- |
| **IN** Events incoming | mdm-onekey-dcr-service:OneKeyResponseStream mdm-veeva-dcr-service:veevaResponseStream | process publisher full change request events in stream | realtime: events stream processing |
Dependent components
====================
| Component | Usage |
| --- | --- |
| [OK DCR Service](/display/GMDM/OK+DCR+Service) | Main component with flow implementation |
| [Veeva DCR Service](https://confluence.pfizer.com/display/GMDM/Veeva+DCR+Service) | Main component with flow implementation |
| [Publisher](/display/GMDM/Publisher) | Events publisher generates incoming events |
| [Hub Store](/display/GMDM/Hub+Store) | DCR and Entities Cache |
---
# Submit Validation Request
**Page ID:** 164469875
**Page Link:** /display/GMDM/Submit+Validation+Request
Description
===========
The process of submitting new validation requests to the OneKey service based on the Reltio change events aggregated in time windows. During this process, new DCRs are created in Reltio.
Flow diagram
============
![](/download/attachments/164469875/OK%20DCR%20Process%20-%20Create%20DCR%20Request%20%28AMER%29.png?version=2&modificationDate=1659956361583&api=v2)
Steps
=====
* Event publisher publishes simple events to $env-internal-onekeyvr-in including HCP\_\*, HCO\_\*, ENTITY\_MATCHES\_CHANGED
* Events are aggregated in a time window (recommended the window length 4 hours) and the last event is returned to the process after the window is closed.
* Simple events are enriched with the Entity data using HUB Cache
* Then, the following checks are executed
+ check if at least one crosswalk create date is equal or above for a given source name and cut off date specified in configuration - section submitVR/crosswalkDecisionTables
+ check if entity attribute values match specified in configuration
+ check if there is no valid DCR created for the entity
+ check if the entity is active
+ check if the OK crosswalk doesn't exist after the full entity retrieval from the HUB cache
+ match category is not 99
+ GetMatches operation from Reltio returns 0 potential matches
* If any check is negative then the process is aborted.
* DCRRequest object is created in Mongo for the flow state recording and generation of the new unique DCR ID for validation request and data tracing.
* The entity is mapped to OK VR Request and it's submitted using API REST method POST /vr/submit.
* If the submission is successful then:
+ *DCRRequest.status* is updated to *SENT* with OK request and response details
+ ***DCR*** entity is created in Reltio and the relation between the processed entity and the DCR entity
- Reltio source name (crosswalk.type): *DCR*
- Reltio relation type: *HCPtoDCR*or *HCOtoDCR*(depending on the object type)
- DCR entity attributes:
| DCR entity attributes | Mapping |
| --- | --- |
| DCRID | OK VR Reqeust Id (cegedimRequestEid) |
| EntityURI | the processed entity URI |
| VRStatus" | "*OPEN"* |
| VRStatusDetail | "*SENT*" |
| CreatedBy | *MDM HUB* |
| SentDate | current time |
* Otherwise *FAILED*status is recorded in *DCRRequest*with an OK error response.
+ *DCRRequest.status* is updated to *FAILED* with OK request and exception response details
+ ***DCR*** entity is created in Reltio and the relation between the processed entity and the DCR entity
- Reltio source name (crosswalk.type): *DCR*
- Reltio relation type: *HCPtoDCR* or *HCOtoDCR* (depending on the object type)
- DCR entity attributes:
| DCR entity attributes | Mapping |
| --- | --- |
| DCRID | OK VR Reqeust Id (cegedimRequestEid) |
| EntityURI | the processed entity URI |
| VRStatus | "*CLOSED"* |
| VRStatusDetail | "*FAILED*" |
| Comments | ONEKEY service failed [exception details] |
| CreatedBy | *MDM HUB* |
| SentDate | current time |
Triggers
========
| Trigger action | Component | Action | Default time |
| --- | --- | --- | --- |
| **IN** Events incoming | mdm-onekey-dcr-service:OneKeyStream | process publisher simple events in stream | events stream processing with 4h time window events aggregation |
| **OUT** API request | one-key-client:OneKeyIntegrationService.submitValidation | submit VR request to OneKey | invokes API request for each accepted event |
Dependent components
====================
| Component | Usage |
| --- | --- |
| [OK DCR Service](/display/GMDM/OK+DCR+Service) | Main component with flow implementation |
| [Publisher](/display/GMDM/Publisher) | Events publisher generates incoming events |
| [Manager](/display/GMDM/Manager) | Reltio Adapter for getMatches and created operations |
| OneKey Adapter | Submits Validation Request |
| [Hub Store](/display/GMDM/Hub+Store) | DCR and Entities Cache |
Mappings
========
Reltio → OK mapping file: [onkey\_mappings.xlsx](/download/attachments/164469875/onkey_mappings.xlsx?version=1&modificationDate=1611657414000&api=v2)
OK mandatory / required fields: [VR - Business Fields Requirements(Pfizer).xlsx](/download/attachments/164469875/VR%20-%20Business%20Fields%20Requirements%28Pfizer%29.xlsx?version=1&modificationDate=1725884542887&api=v2)
OneKey Documentation
====================
[![](/rest/documentConversion/latest/conversion/thumbnail/510268029/1)](/download/attachments/164469875/Onekey%20WebServices%20-%20Developer%20Guide.docx?version=1&modificationDate=1739460519677&api=v2)
---
# Trace Validation Request
**Page ID:** 164469983
**Page Link:** /display/GMDM/Trace+Validation+Request
Description
===========
The process of tracing the VR changes based on the OneKey VR changes. During this process HUB, DCR Cache is triggered every <T> hour for SENT DCR's and check VR status using OneKey web service. After verification DCR is updated in Reltio or a new Workflow is started in Reltio for the Data Steward manual validation.
Flow diagram
============
![](/download/attachments/164469983/OK%20DCR%20Process%20-%20Validate%20DCR%20Request%281%29.png?version=2&modificationDate=1640093280047&api=v2)
Steps
=====
* Every N  hours OK *VR requests* with status *SENT* are queried in *DCRRequests* store*.*
* For each open requests, its status is checked it OK using REST API method /vr/trace
* The first check is the *VR.rsp.status*attribute, checking if the status is *SUCCESS*
* Next, if the process status (*VR.rsp.results.**processStatus***) is REQUEST\_PENDING\_OKE | REQUEST\_PENDING\_JMS | REQUEST\_PROCESSED **or** OK data export date (*VR.rsp.results.*t*race6CegedimOkcExportDate*) is earlier than 24 hours then the processing of the request is postponed to the next check
+ *exportDate* or ***processStatus***are optional and can be null.
+ The process goes to the next step only if ***processStatus*** is  REQUEST\_RESPONDED | RESPONSE\_SENT
+ The process is blocked to next check only if  t*race6CegedimOkcExportDate*is not null and is earlier than 24h
* If the ***processStatus***is validated and *VR.rsp.results**.**responseStatus***is VAS\_NOT\_FOUND | VAS\_INCOHERENT\_REQUEST | VAS\_DUPLICATE\_PROCESS then DCR is being closed with status *REJECTED*
| DCR entity attributes | Mapping |
| --- | --- |
| VRStatus" | "*CLOSED"* |
| VRStatusDetail | "*REJECTED*" |
| ReceivedDate | current time |
| Comments | *OK.responseComment* |
* Before these 2 next steps, the current Entity status is retrieved from HUB Cache. This is required to check if the entity was merged with OK entity.
+ if ***responseStatus*** is*VAS\_FOUND | VAS\_FOUND\_BUT\_INVALID and* OK crosswalk exists in Reltio entity which value equals to OK validated id (i*ndividualEidValidated or workplaceEidValidated) then* DCR is closed with status *ACCEPTED*.
| DCR entity attributes | Mapping |
| --- | --- |
| VRStatus" | "*CLOSED"* |
| VRStatusDetail | "*ACCEPTED*" |
| ReceivedDate | current time |
| Comments | *OK.responseComment* |
+ if ***responseStatus*** is*VAS\_FOUND | VAS\_FOUND\_BUT\_INVALID but*OK crosswalk doesn't exist in Reltio then Relio DCR Request is created and workflow task is triggered for Data Steward review. DCR status entity is updated with *DS\_ACTION\_REQUIRED* status.
| DCR entity attributes | Mapping |
| --- | --- |
| VRStatus" | "*OPEN"* |
| VRStatusDetail | "DS\_ACTION\_REQUIRED " |
| ReceivedDate | current time |
| Comments | *OK.responseComment* |
+ GET /changeRequests operation is invoked to get a new change request ID and start a new workflow
+ POST /workflow/\_initiate operation is invoked to init new Workflow in Reltio
| Workflow attributes | Mapping |
| --- | --- |
| changeRequest.uri | ChangeRequest Reltio URI |
| changeRequest.changes | Entity URI |
| comment | i*ndividualEidValidated or workplaceEidValidated* |
+ POST /entities?changeRequestId=<id> - operation is invoked to update change request Entity container with DCR Status to Closed, this change is only visible in Reltio once DS accepts the DCR. 
| Body attributes | Mapping |
| --- | --- |
| attributes | ``` "DCRRequests": [ { "value": { "VRStatus": [ { "value": "CLOSED" } ] }, "refEntity": { "crosswalks": [ { "type": "configuration/sources/DCR", "value": "$requestId", "dataProvider": false, "contributorProvider": true }, { "type": "configuration/sources/DCR", "value": "$requestId_REF", "dataProvider": true, "contributorProvider": false } ] }, "refRelation": { "crosswalks": [ { "type": "configuration/sources/DCR", "value": "$requestId_REF" } ] } } ] ``` |
| crosswalks | ``` "crosswalks": [ { "type": "configuration/sources/<source crosswalk>", "value": "<source value>", "dataProvider": false, "contributorProvider": true, "deleteDate": "" }, { "type": "configuration/sources/DCR", "value": "$requestId_CR", "dataProvider": true, "contributorProvider": false, "deleteDate": "" } ] ``` |
Triggers
========
| Trigger action | Component | Action | Default time |
| --- | --- | --- | --- |
| **IN** Timer (cron) | mdm-onekey-dcr-service:TraceVRService | query mongo to get all SENT DCR's related to OK\_VR process | every <T> hour |
| **OUT** API request | one-key-client:OneKeyIntegrationService.traceValidation | trace VR request to OneKey | invokes API request for each DCR |
Dependent components
====================
| Component | Usage |
| --- | --- |
| [OK DCR Service](/display/GMDM/OK+DCR+Service) | Main component with flow implementation |
| [Manager](/display/GMDM/Manager) | Reltio Adapter for GET /changeRequests and POST /workflow/\_initiate operations |
| OneKey Adapter | TraceValidation Request |
| [Hub Store](/display/GMDM/Hub+Store) | DCR and Entities Cache |
---
# PforceRx DCR flows
**Page ID:** 209949183
**Page Link:** /display/GMDM/PforceRx+DCR+flows
Description
===========
MDM HUB exposes Rest API to create and check the status of DCR. The process is responsible for creating DCRs in Reltio and starting Change Requests Workflow DCRs created in Reltio or creating the DCRs (submitVR operation) in ONEKEY. DCR requests can be routed to an external MDM HUB instance handling the requested country. The action is transparent to the caller. During this process, the communication to IQVIA OneKey VR API / Reltio API is established. The routing decision depends on the market, operation type, or changed profile attributes.
Reltio API:  createEntity (with ChangeReqest) operation is executed to create a completely new entity in the new Change Request in Reltio. attributesUpdate (with ChageRequest) operation is executed after calculation of the specific changes on complex or simple attributes on existing entity - this also creates a new Change Request.  Start Workflow operation is requested at the end, this starts the Wrofklow for the DCR in Reltio so the change requests are started in the Reltio Inbox for Data Steward review.
IQVIA API: SubmitVR operation is executed to create a new Validation Request. The TraceVR operation is executed to check the status of the VR in OneKey.
All DCRs are saved in the dedicated collection in HUB Mongo DB, required to gather metadata and trace the changes for each DCR request. The DCR statuses are updated by consuming events generated by Reltio or periodic query action of open DCRs in OneKey
The Data Steward can decide to route a DCR to IQVIA as well - some changes can be suggested by the DS using the "Suggest" operation in Reltio and "Send to Third Party Validation" button, the process "Data Steward OK Validation Request" is processing these changes and sends them to the OneKey service.
The below diagram presents an overview of the entire process. Detailed descriptions are available in the separated subpages.
**API doc URL**: <https://api-emea-nprod-gbl-mdm-hub.pfizer.com/api-dcr-spec-emea-dev/swagger-ui/index.html>
Flow diagram
============
### DCR Service High-Level Architecture
![](/download/attachments/209949183/image-2024-4-4_17-19-31.png?version=1&modificationDate=1712243971410&api=v2)
### DCR HUB Logical Architecture
![](/download/attachments/209949183/image-2024-4-4_17-53-51.png?version=1&modificationDate=1712246031487&api=v2)
Model diagram
=============
![](/download/attachments/209949183/image-2023-11-16_16-21-2.png?version=1&modificationDate=1700148063087&api=v2)
Flows:
======
* [Create DCR](/display/GMDM/Create+DCR)
+ The client call API Post /dcr method and pass the request in JSON format to MDM HUB DCR service
+ The request is validated against the following rules:
- mandatory fields are set
- reference object HCP,HCO are available in Reltio
- referenced attributes like specialties, addresses are in the changed object
+ The service evaluates the target system based on country, operation type (create, update), changed attributes. The process is controlled by the decision table stored in the config.
+ The DCR is created in the target system through the API
+ The result is stored in the registry. DCR information entity is created in Reltio for tracking.
+ The status with created DCR object ids are returned in response to the Client
* [Get DCR status](/display/GMDM/Get+DCR+status)
+ The client calls GET /dcr/\_status method
+ The DCR service queries DCR registry in Mongo and returns the status to the Client.
+ There are processes updating dcr status in the registry:
- DCR change events are generated by Reltio when DCR is accepted or rejected by DS. Events are processed by the service.
* [Reltio: process DCR Change Events](/display/GMDM/Reltio%3A+process+DCR+Change+Events)
+ DCR change events are generated by Reltio when DCR is accepted or rejected by DS. Events are processed by the service.
* [OneKey: process DCR Change Events](/display/GMDM/OneKey%3A+process+DCR+Change+Events)
+ DCR change events are generated by the OneKey service when DCR is accepted or rejected by DS. Events are processed by the service.
* [OneKey: generate DCR Change Events (traceVR)](/pages/viewpage.action?pageId=209950500)
+ Every x configured hours the OneKey status method is queried to get status for open validation requests.
* [Reltio: create DCR method - direct](/display/GMDM/Reltio%3A+create+DCR+method+-+direct)
+ direct API method that creates DCR in Reltio (contains mapping and logic description)
* [OneKey: create DCR method (submitVR) - direct](/display/GMDM/OneKey%3A+create+DCR+method+%28submitVR%29+-+direct)
+ direct API method that creates DCR in OneKey - executes the submitVR operation (contains mapping and logic description)
Triggers
========
Described in the separated sub-pages for each process.
Dependent components
====================
Described in the separated sub-pages for each process.
---
# Create DCR
**Page ID:** 209949185
**Page Link:** /display/GMDM/Create+DCR
Description
===========
The process creates change requests received from PforceRx Client and sends the DCR to the specified target service - Reltio, OneKey or Veeva OpenData (VOD). DCR is created in the system and then processed by the data stewards. The status is asynchronously updated by the HUB processes, Client represents the DCR using a unique *extDCRRequestId* value. Using this value Client can check the status of the DCR ([Get DCR status](/display/GMDM/Get+DCR+status)).
Flow diagram
============
![](/download/attachments/209949185/image-2023-12-12_13-37-9.png?version=1&modificationDate=1702384629563&api=v2)
*Source: [Lucid](https://lucid.app/lucidchart/4143a333-574e-4a47-b305-ce450202ce6a/edit?view_items=z2yKmaLK%2Fumrvac7R0EffxYeaXE%3D&invitationId=inv_5f6a4de0-884a-4859-a13b-e9db321eb4b6)*
![](/download/attachments/209949185/image-2023-12-12_13-43-54.png?version=1&modificationDate=1702385035043&api=v2)
*Source: [Lucid](https://lucid.app/lucidchart/4143a333-574e-4a47-b305-ce450202ce6a/edit?view_items=z2yKmaLK%2Fumrvac7R0EffxYeaXE%3D&invitationId=inv_5f6a4de0-884a-4859-a13b-e9db321eb4b6)*
DCR Service component perspective
=================================
![](https://documents.lucid.app/documents/b0fc33c3-3eda-4fdf-b480-3166a42e1e4d/pages/0_0?a=1264&x=-26&y=28&w=2332&h=1144&store=1&accept=image%2F*&auth=LCA%208e4be02488387f716c2f4b3fab09af06e0fcf25e-ts%3D1642496662)
Steps
=====
1. Clients execute the API POST /dcr request
2. Kong receives requests and handles authentication
3. If the authentication succeeds the request is forwarded to the dcr-service-2 component,
4. DCR Service checks permissions to call this operation and the correctness of the request, then the flow is started and the following steps are executed:
1. Parse and validate the dcr request. The validation logic checks the following:
1. Check if the list of `DCRRequests` contains unique `extDCRRequestId`.
1. Requests that are duplicate will be rejected with the error message - *"Found duplicated request(s)"*
2. For each `DCRRequest` in the input list execute the following checks:
1. Users can define the following number of entities in the Request:
1. at least one entity has to be defined, otherwise, the request will be rejected with an error message - *"No entities found in the request"*
2. single HCP
3. singe HCO
4. singe HCP with single HCO
5. two HCOs
2. Check if the main reference objects exist in Reltio for **update** and **delete** action
1. `HCP.refId` or `HCO.refId`, user have to specify one of:
1. ***`CrosswalkTargetObjectId`*** - then the entity is retrieved from Reltio using get entity by crosswalk operation
2. ***`EntityURITargetObjectId`*** - then the entity is retrieved from Reltio using get entity by uri operation
3. ***`PfizerCustomerIdTargetObjectId`*** - then the entity is retrieved from Reltio using search operation by the `PfizerGlobalCustomerID`
3. Attributes validation:
1. Simple attributes - like firstName/lastName e.t.c
1. for ***update*** action on the main object:
1. if the input parameter is defined with an empty value - **""** - this will result in the removal of the target attribute
2. if the input parameter is defined with a non-empty value - this will result in the update of the target attribute
2. Nested attributes - like Specialties/Addresses e.t.c
1. for each attribute, the user has to define the **refId** to uniquely identify the attribute
1. For action "***update***" - if the **refId** is not found in the target object request will be rejected with a detailed error message
2. For action "***insert***" - the refId is not required - new reference attribute will be added to the target object
4. Changes validation:
1. If the validation detected 0 changes (during comparison of applying changes and the target entity) -  the request is rejected with an error message - *"No changes detected"*
2. Evaluate dcr service (based on the decision table config)
1. The following decision table is defined to choose the target service
1. LIST OF the following combination of attributes:
| attribute | description |
| --- | --- |
| ``` userName ``` | the user name that executes the request |
| ``` sourceName ``` | the source name of the Main object |
| ``` country ``` | the county defined in the request |
| ``` operationType ``` | the operation type for the Main object ``` { insert, update, delete } ``` |
| ``` affectedAttributes ``` | the list of attributes that the user is changing |
| ``` affectedObjects ``` | ``` { HCP, HCO, HCP_HCO } ``` |
| | RESULT →  TargetType {Reltio, OneKey, Veeva} |
2. Each attribute in the configuration is optional.
3. The decision table is making the validation based on the input request and the main object- the main object is HCP, if the HCP is empty then the decision table is checking HCO.
4. The result of the decision table is the `TargetType`, the routing to the Reltio MDM system, OneKey or Veeva service.
3. Execute target service (reltio/onekey/veeva)
1. [Reltio: create DCR method - direct](/display/GMDM/Reltio%3A+create+DCR+method+-+direct)
2. [OneKey: create DCR method (submitVR) - direct](/display/GMDM/OneKey%3A+create+DCR+method+%28submitVR%29+-+direct)
3. [Veeva: create DCR method (storeVR)](/pages/viewpage.action?pageId=379332642)
4. Create DCR in Reltio and save DCR in DCR Registry
* If the submission is successful then:
+ ***DCR***entity is created in Reltio and the relation between the processed entity and the DCR entity
- Reltio source name (crosswalk.type): *DCR*
- Reltio relation type: *HCPtoDCR*or *HCOtoDCR*(depending on the object type)
* for "**create**" and "**delete**" operation the Relation have to be created between objects
* if this is just the "**insert**" operation the Relation will be created after the acceptance of the Change Request in Reltio - [Reltio: process DCR Change Events](/display/GMDM/Reltio%3A+process+DCR+Change+Events)
- DCR entity attributes once sent to *OneKey*
| DCR entity attributes | Mapping |
| --- | --- |
| DCRID | ***extDCRRequestId*** |
| EntityURI | the processed entity URI |
| VRStatus | "*OPEN"* |
| VRStatusDetail | "*SENT\_TO\_OK"* |
| CreatedBy | *MDM HUB* |
| SentDate | current time |
| CreateDate | current time |
| CloseDate | if REJECTED | ACCEPTED -> current time |
| dcrType | evaluate based on config: ``` dcrTypeRules: - type: CR0 size: 1 action: insert entity: com.pfizer.mdm.api.dcr2.HCP ``` |
- DCR entity attributes once sent to *Veeva*
| *DCR entity attributes* | *Mapping* |
| --- | --- |
| *DCRID* | ***extDCRRequestId*** |
| *EntityURI* | *the processed entity URI* |
| *VRStatus* | *"OPEN"* |
| *VRStatusDetail* | *"SENT\_TO\_VEEVA"* |
| *CreatedBy* | *MDM HUB* |
| *SentDate* | *current time* |
| *CreateDate* | *current time* |
| *CloseDate* | *if REJECTED | ACCEPTED -> current time* |
| *dcrType* | *evaluate based on config:* ``` dcrTypeRules: - type: CR0 size: 1 action: insert entity: com.pfizer.mdm.api.dcr2.HCP ``` |
- DCR entity attributes once sent to *Reltio* → action is passed to DS and workflow is started.
| DCR entity attributes | Mapping |
| --- | --- |
| DCRID | ***extDCRRequestId*** |
| EntityURI | the processed entity URI |
| VRStatus | "*OPEN"* |
| VRStatusDetail | "DS\_ACTION\_REQUIRED " |
| CreatedBy | *MDM HUB* |
| SentDate | current time |
| CreateDate | current time |
| CloseDate | if REJECTED | ACCEPTED -> current time |
| dcrType | evaluate based on config: ``` dcrTypeRules: - type: CR0 size: 1 action: insert entity: com.pfizer.mdm.api.dcr2.HCP ``` |
+ *Mongo Update: `DCRRequest.status`* is updated to *SENT*with OneKey or Veeva request and response details or DS\_ACTION\_REQURIED with all Reltio details
* Otherwise *FAILED*status is recorded in `DCRRequest`with a detailed error message.
+ *Mongo Update:  `DCRRequest.status`* is updated to *FAILED* with all required attributes, request, and exception response details
5. Initialize Workflow in Reltio (only requests that `TargetType`is Reltio)
1. POST /workflow/\_initiate operation is invoked to init new Workflow in Reltio
| Workflow attributes | Mapping |
| --- | --- |
| changeRequest.uri | ChangeRequest Reltio URI |
| changeRequest.changes | Entity URI |
6. Then Auto close logic is invoked to evaluate whether DCR request meets conditions to be auto accepted or auto rejected. Logic is based on decision table `PreCloseConfig`. If `DCRRequest.country` is contained in `PreCloseConfig.acceptCountries` or `PreCloseConfig.rejectCountries` then DCR is accepted or rejected respectively.
7. return DCRResponse to Client - During the flow, DCRRespone may be returned to Client with the specific ***errorCode*** or ***requestStatus***. The description for all response codes is presented on this page: [Get DCR status](/display/GMDM/Get+DCR+status)
Triggers
========
| Trigger action | Component | Action | Default time |
| --- | --- | --- | --- |
| REST call | DCR Service: POST /dcr | create DCRs in the Reltio, OneKey or Veeva system | API synchronous requests - realtime |
Dependent components
====================
| Component | Usage |
| --- | --- |
| [DCR Service](/display/GMDM/DCR+Service) | Main component with flow implementation |
| [OK DCR Service](/display/GMDM/OK+DCR+Service) | OneKey Adapter - API operations |
| [Veeva DCR Service](/display/GMDM/Veeva+DCR+Service) | Veeva Adapter - API operations and S3/SFTP communication |
| [Manager](/display/GMDM/Manager) | Reltio Adapter - API operations |
| [Hub Store](/display/GMDM/Hub+Store) | DCR and Entities Cache |
---
# DCR state change
**Page ID:** 218438617
**Page Link:** /display/GMDM/DCR+state+change
Description
===========
The following diagram represents the DCR state changes. DCR object stat is saved in HUB and in Reltio DCR entity object. The state of the DCR is changed based on the Reltio/IQVIA/Veeva Data Steward action.
Flow diagram
============
![](/download/attachments/218438617/image-2024-2-7_9-57-8.png?version=1&modificationDate=1707296228470&api=v2)
Steps
=====
1. DCR is created (OPEN)  - [Create DCR](/display/GMDM/Create+DCR)
1. DCR is sent to Reltio, OneKey or Veeva
1. When sent to Reltio
1. Pre Close logic is invoked to auto accept (PRE\_ACCEPT) or auto reject (PRE\_REJECT) DCR
2. Reltio Data Steward process the DCR - [Reltio: process DCR Change Events](/display/GMDM/Reltio%3A+process+DCR+Change+Events)
2. OneKey Data Steward process the DCR - [OneKey: process DCR Change Events](/display/GMDM/OneKey%3A+process+DCR+Change+Events)
3. Veeva Data Steward process the DCR - [Veeva: process DCR Change Events](/display/GMDM/Veeva%3A+process+DCR+Change+Events)
[Data Steward DCR status change perspective](#DCRstatechange-DSstatuschangeperspective)
=======================================================================================
![](/download/attachments/218438617/image-2024-4-4_17-1-46.png?version=1&modificationDate=1712242906987&api=v2)
Transaction Log
===============
There are the following main assumptions regarding the transaction log in DCR service:
* **Main transaction**
+ The user sends to the DCR service list of the DCR Requests and receives the list of the DCR Responses
- Transaction service generates the transaction ID for the input request - this is used as the correlation ID for each separated DCR Request in the list
- Transaction service save:
* METADATA
+ main transaction ID
+ userName
+ extDCRRequestIds (list of all)
* BODY
+ the DCR Requests list and the DCR Response List
* **State change transaction**
+ DCR object state may change depending on the DS decision, for each state change (represented as a green box in the above diagram) the transaction is saved with the following attributes:
- Transaction METADATA
* main transaction ID
* extDCRRequestId
* dcrRequestId
* Reltio:
+ VRStatus
+ VRStatusDetail
* HUB:
+ DCRRequestStatusDetails
* optionally:
+ errorMessage
+ ```
errorCode
```
- Transaction BODY:
* Input Event
**Log appenders:**
* Kafka Transaction appender - saves whole events(metadata+body) to Kafka - data presented in the Kibana Dashboard <link TODO>
* Simple Transaction logger - saves the transactions details to the file in the following format:
+ {ID}    {extDCRRequestId}   {dcrRequestId}   {VRStatus}   {VRStatusDetail}   {DCRRequestStatusDetails}   {errorCode}   {errorMessage}
Triggers
========
| Trigger action | Component | Action | Default time |
| --- | --- | --- | --- |
| REST call | DCR Service: POST /dcr | create DCRs in the Reltio system or in OneKey | API synchronous requests - realtime |
| **IN** Events incoming | dcr-service-2:DCRReltioResponseStream | process publisher full change request events in the stream | realtime: events stream processing |
| **IN** Events incoming | dcr-service-2:DCROneKeyResponseStream | process publisher full change request events in the stream | realtime: events stream processing |
| **IN** Events incoming | dcr-service-2:DCRVeevaResponseStream | process publisher full change request events in the stream | realtime: events stream processing |
Dependent components
====================
| Component | Usage |
| --- | --- |
| [DCR Service](https://confluence.pfizer.com/display/GMDM/DCR+Service) | Main component with flow implementation |
| [OK DCR Service](https://confluence.pfizer.com/display/GMDM/OK+DCR+Service) | OneKey Adapter  - API operations |
| [Veeva DCR Service](/display/GMDM/Veeva+DCR+Service) | Veeva Adapter  - API operations |
| [Manager](https://confluence.pfizer.com/display/GMDM/Manager) | Reltio Adapter  - API operations |
| [Hub Store](https://confluence.pfizer.com/display/GMDM/Hub+Store) | DCR and Entities Cache |
---
# Get DCR status
**Page ID:** 209949187
**Page Link:** /display/GMDM/Get+DCR+status
Description
===========
The client creates DCRs in Reltio, OneKey or Veeva OpenData using the [Create DCR](/display/GMDM/Create+DCR) operation. The status is then asynchronously updated in the DCR Registry. The operation retrieves the current status of the DCRs that the updated date is between '`updateFrom`' and '`updateTo`' input parameters. PforceRx first asks what DCRs have been changed since last time they checked (usually 24h) and then iterate for each DCR they get detailed info.
Flow diagram
============
![](/download/attachments/209949187/New%20DCR%20Process%20FLOWS%20-%20Get%20DCR%20status.png?version=1&modificationDate=1637924234933&api=v2),
![](/download/attachments/209949187/image-2023-12-12_16-41-14.png?version=1&modificationDate=1702395674717&api=v2)
Source: [Lucid](https://lucid.app/lucidchart/4143a333-574e-4a47-b305-ce450202ce6a/edit?viewport_loc=-1025%2C-379%2C4396%2C2439%2CUyGmrIGRrgqf&invitationId=inv_5f6a4de0-884a-4859-a13b-e9db321eb4b6)
##### Dependent flows:
1. The DCRRegistry is enriched by the DCR events that are generated by Reltio - the flow description is here - [Reltio: process DCR Change Events](/display/GMDM/Reltio%3A+process+DCR+Change+Events)
2. The DCRRegistry is enriched by the DCR events generated in OneKey DCR service component - after submitVR operation is invoked to ONEKEY, each DCR is traced asynchronously in this process - [OneKey: process DCR Change Events](/display/GMDM/OneKey%3A+process+DCR+Change+Events)
3. The DCRRegistry is enriched by the DCR events generated in Veeva OpenData DCR service component - after submitVR operation is invoked to VEEVA, each DCR is traced asynchronously in this process - [Veeva: process DCR Change Events](/display/GMDM/Veeva%3A+process+DCR+Change+Events)
Steps
=====
##### Status
There are the following request statuses that users may receive during [Create DCR](/display/GMDM/Create+DCR) operation or during checking the updated status using GET `/dcr/_status` operation described below:
| RequestStatus | DCRStatus | Internal Cache status | Description |
| --- | --- | --- | --- |
| REQUEST\_ACCEPTED | CREATED | SENT\_TO\_OK | DCR was sent to the ONEKEY system for validation and pending the processing by Data Steward in the system |
| REQUEST\_ACCEPTED | CREATED | SENT\_TO\_VEEVA | DCR was sent to the VEEVA system for validation and pending the processing by Data Steward in the system |
| REQUEST\_ACCEPTED | CREATED | DS\_ACTION\_REQUIRED | DCR is pending Data Steward validation in Reltio, waiting for approval or rejection |
| REQUEST\_ACCEPTED | CREATED | OK\_NOT\_FOUND | Used when ONEKEY profile was not found after X retries |
| REQUEST\_ACCEPTED | CREATED | VEEVA\_NOT\_FOUND | Used when VEEVA profile was not found after X retries |
| REQUEST\_ACCEPTED | CREATED | WAITING\_FOR\_ETL\_DATA\_LOAD | Used when waiting for actual data profile load from 3rd Party to appear in Reltio |
| REQUEST\_ACCEPTED | ACCEPTED | ACCEPTED | Data Steward accepted the DCR, changes were applied |
| REQUEST\_ACCEPTED | ACCEPTED | PRE\_ACCEPTED | PreClose logic was invoked and automatically accepted DCR according to decision table in PreCloseConfig |
| REQUEST\_REJECTED | REJECTED | REJECTED | Data Steward rejected the changes presented in the Change Request |
| REQUEST\_REJECTED | REJECTED | PRE\_REJECTED | PreClose logic was invoked and automatically rejected DCR according to decision table in PreCloseConfig |
| REQUEST\_FAILED | - | FAILED | DCR requests failed due to: validation error/ unexpected error e.t.d - details in the errorCode and errorMessage |
##### Error codes:
There are the following classes of exception that users may receive during [Create DCR](/display/GMDM/Create+DCR) operation:
| Class | errorCode | Description | HTTP code |
| --- | --- | --- | --- |
| 1 | DUPLICATE\_REQUEST | request rejected - extDCRRequestId  is registered - this is a duplicate request | 403 |
| 2 | NO\_CHANGES\_DETECTED | entities are the same (request is the same) - no changes | 400 |
| 3 | VALIDATION\_ERROR | ref object does not exist (not able to find HCP/HCO target object | 404 |
| 3 | VALIDATION\_ERROR | ref attribute does not exist - not able to find nested attribute in the target object | 400 |
| 3 | VALIDATION\_ERROR | wrong number of HCP/HCO entities in the input request | 400 |
1. Clients execute the API GET`/dcr/_status` request
2. Kong receives requests and handles authentication
3. If the authentication succeeds the request is forwarded to the dcr-service-2 component,
4. DCR Service checks permissions to call this operation and the correctness of the request, then the flow is started and the following steps are executed
1. Query on mongo is executed to get all DCRs matching input parameters:
1. updateFrom (date-time) - DCR last update from - *DCRRequestDetails.status.changeDate*
2. updateTo (date-time) - DCR last update to - *DCRRequestDetails.status.changeDate*
3. limit (int) the maximum number of results returned through API - the recommended value is 25. The max value for a single request is 50.
4. offset(int) - result offset - the parameter used to query through results that exceeded the limit.
2. Resulted values are aggregated and returned to the Client.
3. The client receives the List<DCRResposne> body.
Triggers
========
| Trigger action | Component | Action | Default time |
| --- | --- | --- | --- |
| REST call | DCR Service: GET`/dcr/_status` | get status of created DCRs. Limit the results using query parameters like dates and offset | API synchronous requests - realtime |
Dependent components
====================
| Component | Usage |
| --- | --- |
| [DCR Service](https://confluence.pfizer.com/display/GMDM/DCR+Service) | Main component with flow implementation |
| [Hub Store](https://confluence.pfizer.com/display/GMDM/Hub+Store) | DCR and Entities Cache |
---
# OneKey: create DCR method (submitVR) - direct
**Page ID:** 209949294
**Page Link:** /display/GMDM/OneKey%3A+create+DCR+method+%28submitVR%29+-+direct
Description
===========
Rest API method exposed in the [OK DCR Service](/display/GMDM/OK+DCR+Service) component responsible for submitting the VR to OneKey
Flow diagram
============
![](/download/attachments/209949294/New%20DCR%20Process%20-%20OneKey%20create%20DCR%20method%20%28submitVR%29%20-%20direct%281%29.png?version=1&modificationDate=1641209972037&api=v2)
Steps
=====
1. Receive the API request
2. Validate - check if the onekey crosswalk exists once there is an update on the profile, otherwise reject the request
3. The DCR is mapped to OK VR Request and it's submitted using API REST method POST /vr/submit. (mapping described below)
1. If the submission is successful then:
* *DCRRequest*i updated to *SENT\_TO\_OK*with OK request and response details. DCRRegistryONEKEY collection in saved for tracing purposes. The process that reads and check ONEKEY VRs is described here: [OneKey: generate DCR Change Events (traceVR)](/pages/viewpage.action?pageId=209950500)
2. Otherwise *FAILED*status is recorded and the response is returned with an OK error response
Mapping
=======
[VR - Business Fields Requirements\_UK.xlsx](/download/attachments/209949294/VR%20-%20Business%20Fields%20Requirements_UK.xlsx?version=1&modificationDate=1648131143387&api=v2) - file that contains VR UK requirements and mapping to IQVIA model
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
| | | | | | | | |
| --- | --- | --- | --- | --- | --- | --- | --- |
| **HUB** | | | | | **ONEKEY** | | |
| | ***attributes*** | ***attributes*** | ***codes*** | | ***mandatory*** | ***attributes*** | ***values*** |
| **HCO** | | | | | | | |
| | | | | | Y | entityType | WORKPLACE |
| | | | | | Y | validation.clientRequestId | HUB\_GENERATED\_ID |
| | | | | | Y | validation.process | Q |
| | | | | | Y | validation.requestDate | 1970-01-01T00:00Z |
| | | | | | Y | validation.callDate | 1970-01-01T00:00Z |
| **attributes** | | | | | Y | validation.requestProcess | I |
| | extDCRComment | | | | | validation.requestComment | |
| | | | | | | | |
| | country | | | | Y | isoCod2 | |
| | | | | | | | |
| | | | | | | | |
| | reference Entity | crosswalk | ONEKEY | | | workplace.workplaceEid | |
| | | | | | | | |
| | name | | | | | workplace.usualName | |
| | | | | | | workplace.officialName | |
| | otherHCOAffiliations | parentUsualName | | | | workplace.parentUsualName | |
| | subTypeCode | | COTFacilityType (TET.W.\*) | | | workplace.typeCode | |
| | ~~typeCode~~ | no value in PFORCERX | ~~HCOSubType~~ ~~(LEX.W.\*)~~ | | | ~~workplace.activityLocationCode~~ | |
| | addresses | | | | | | |
| | | sourceAddressId | | | | N/A | |
| | | addressType | | | | N/A | |
| | | addressLine1 | | | | address.longLabel | |
| | | addressLine2 | | | | address.longLabel2 | |
| | | addressLine3 | | | | N/A | |
| | | stateProvince | AddressState (DPT.W.\*) | | | address.countyCode | |
| | | city | | | Y | address.city | |
| | | zip | | | | address.longPostalCode | |
| | | country | | | Y | address.country | |
| | | rank | | | | get address with rank=1 | |
| | emails | | | | | | |
| | | type | | | | N/A | |
| | | email | | | | workplace.email | |
| | | rank | | | | get email with rank=1 | |
| | otherHCOAffiliations | | | | | | |
| | | type | | | | N/A | |
| | | rank | | | | get affiliation with rank=1 | |
| | reference Entity | otherHCOAffiliations reference entity onekeyID | ONEKEY | | | workplace.parentWorkplaceEid | |
| | phones | | | | | | |
| | | type | contains FAX | | | | |
| | | number | | | | workplace.telephone | |
| | | rank | | | | get phone with rank=1 | |
| | | | | | | | |
| | | type | not contains FAX | | | | |
| | | number | | | | workplace.fax | |
| | | rank | | | | get phone with rank=1 | |
| HCP | | | | | | | |
| | | | | | Y | entityType | ACTIVITY |
| | | | | | Y | validation.clientRequestId | HUB\_GENERATED\_ID |
| | | | | | Y | validation.process | Q |
| | | | | | Y | validation.requestDate | 1970-01-01T00:00Z |
| | | | | | Y | validation.callDate | 1970-01-01T00:00Z |
| **attributes** | | | | | Y | validation.requestProcess | I |
| | extDCRComment | | | | | validation.requestComment | |
| | | | | | | | |
| | country | | | | Y | isoCod2 | |
| | | | | | | | |
| | | | | | | | |
| | reference Entity | crosswalk | ONEKEY | | | individual.individualEid | |
| | | | | | | | |
| | firstName | | | | | individual.firstName | |
| | lastName | | | | Y | individual.lastName | |
| | middleName | | | | | individual.middleName | |
| | typeCode | | | | | N/A | |
| | subTypeCode | | HCPSubTypeCode (TYP..\*) | | | individual.typeCode | |
| | title | | HCPTitle (TIT.\*) | | | individual.titleCode | |
| | prefix | | HCPPrefix (APP.\*) | | | individual.prefixNameCode | |
| | suffix | | | | | N/A | |
| | gender | | Gender (.\*) | | | individual.genderCode | |
| | specialties | | | | | | |
| | | typeCode | HCPSpecialty (SP.W.\*) | | | individual.speciality1 | |
| | | type | | | | N/A | |
| | | rank | | | | get speciality with rank=1 | |
| | | typeCode | HCPSpecialty (SP.W.\*) | | | individual.speciality2 | |
| | | type | | | | N/A | |
| | | rank | | | | get speciality with rank=2 | |
| | | typeCode | HCPSpecialty (SP.W.\*) | | | individual.speciality3 | |
| | | type | | | | N/A | |
| | | rank | | | | get speciality with rank=3 | |
| | addresses | | | | | | |
| | | sourceAddressId | | | | N/A | |
| | | addressType | | | | N/A | |
| | | addressLine1 | | | | address.longLabel | |
| | | addressLine2 | | | | address.longLabel2 | |
| | | addressLine3 | | | | N/A | |
| | | stateProvince | AddressState (DPT.W.\*) | | | address.countyCode | |
| | | city | | | Y | address.city | |
| | | zip | | | | address.longPostalCode | |
| | | country | | | Y | address.country | |
| | | rank | | | | get address with rank=1 | |
| | identifiers | | | | | | |
| | | type | | | | N/A | |
| | | id | | | | N/A | |
| | phones | | | | | | |
| | | type | | | | N/A | |
| | | number | | | | individual.mobilePhone | |
| | | rank | | | | get phone with rank=1 | |
| | emails | | | | | | |
| | | type | | | | N/A | |
| | | email | | | | individual.email | |
| | | rank | | | | get phone with rank=1 | |
| | | | | | | | |
| --- | --- | --- | --- | --- | --- | --- | --- |
| | ~~contactAffiliations~~ | no value in PFORCERX | | | | | |
| | | ~~type~~ | ~~RoleType~~ ~~(TIH.W.\*)~~ | | | ~~activity.role~~ | |
| | | ~~primary~~ | | | | ~~N/A~~ | |
| | | ~~rank~~ | | | | ~~get affiliation with rank=1~~ | |
| | contactAffiliations reference Entity | crosswalks | ONEKEY | | | workplace.workplaceEid | |
| HCP & HCO | | | | | | | |
| | | | | | Y | entityType | ACTIVITY |
| | For HCP full mapping check the HCP section above | | | | Y | validation.clientRequestId | HUB\_GENERATED\_ID |
| | For HCO full mapping check the HCO section above | | | | Y | validation.process | Q |
| | | | | | Y | validation.requestDate | 1970-01-01T00:00Z |
| | | | | | Y | validation.callDate | 1970-01-01T00:00Z |
| **attributes** | | | | | Y | validation.requestProcess | I |
| | extDCRComment | | | | | validation.requestComment | |
| | | | | | | | |
| | country | | | | Y | isoCod2 | |
| | addresses | | | | | | |
| | | If the HCO address exists map to ONEKEY address | | | | address (mapping HCO) | |
| | | else | | | | | |
| | | If the HCP address exists map to ONEKEY address | | | | address (mapping HCP) | |
| | ~~contactAffiliations~~ | no value in PFORCERX | | | | | |
| | | ~~type~~ | ~~RoleType~~ ``` (TIH.W.*) ``` | | | ~~activity.role~~ | |
| | | ~~primary~~ | | | | ~~N/A~~ | |
| | | ~~rank~~ | | | | ~~get affiliation with rank=1~~ | |
Triggers
========
| Trigger action | Component | Action | Default time |
| --- | --- | --- | --- |
| REST call | DCR Service: POST /dcr | create DCRs in the ONEKEY | API synchronous requests - realtime |
Dependent components
====================
| Component | Usage |
| --- | --- |
| [DCR Service 2](/display/GMDM/DCR+Service+2) | Main component with flow implementation |
| [Hub Store](https://confluence.pfizer.com/display/GMDM/Hub+Store) | DCR and Entities Cache |
---
# OneKey: generate DCR Change Events (traceVR)
**Page ID:** 209950500
**Page Link:** /pages/viewpage.action?pageId=209950500
Description
===========
This process is triggered after the DCR was routed to Onekey based on the decision table configuration. The process of tracing the VR changes is based on the OneKey VR changes. During this process HUB, DCR Cache is triggered every <T> hour for SENT DCR's and check VR status using OneKey web service. After verification, the DCR Change event is generated. The DCR event is processed in the [OneKey: process DCR Change Events](/display/GMDM/OneKey%3A+process+DCR+Change+Events) and the DCR is updated in Reltio with Accepted or Rejected status.
Flow diagram
============
![](/download/attachments/209950500/New%20DCR%20Process%20-%20OneKey%20generate%20DCR%20Change%20Events%20%28traceVR%29%282%29.png?version=1&modificationDate=1641210050077&api=v2)
Steps
=====
* Every N  hours OK *VR requests* with status *SENT*are queried in *DCRRegistryONEKEY* store*.*
* For each open requests, its status is checked it OK using REST API method /vr/trace
* The first check is the *VR.rsp.status*attribute, checking if the status is *SUCCESS*
* Next, if the process status (*VR.rsp.results.**processStatus***) is REQUEST\_PENDING\_OKE | REQUEST\_PENDING\_JMS | REQUEST\_PROCESSED **or** OK data export date (*VR.rsp.results.*t*race6CegedimOkcExportDate*) is earlier than 24 hours then the processing of the request is postponed to the next check
+ *exportDate* or ***processStatus***are optional and can be null.
+ The process goes to the next step only if ***processStatus***is  REQUEST\_RESPONDED | RESPONSE\_SENT
+ The process is blocked to next check only if  t*race6CegedimOkcExportDate*is not null and is earlier than 24h
* If the ***processStatus***is validated and *VR.rsp.results**.**responseStatus***is VAS\_NOT\_FOUND | VAS\_INCOHERENT\_REQUEST | VAS\_DUPLICATE\_PROCESS then OneKeyDCREvent is being generated with status *REJECTED*
| OneKeyChangeRequest attributes | Mapping |
| --- | --- |
| vrStatus | "*CLOSED"* |
| vrStatusDetail | "*REJECTED*" |
| traceResponseReceivedDate | current time |
| oneKeyComment | *OK.responseComment* |
* Next.
+ if ***responseStatus***is*VAS\_FOUND | VAS\_FOUND\_BUT\_INVALID* then OneKeyDCREvent is being generated with status *ACCEPTED*. ( now the new ONEKEY profile will be loaded to Reltio using ETL data load. The [OneKey: process DCR Change Events](/display/GMDM/OneKey%3A+process+DCR+Change+Events) is processing this events ad checks the Reltio if the ONEKEY is created and PfizerCustomerGlobalId is assigned, this process will wait until ONEKEY is in Reltio so the client received the ACCEPTED DCR only after this condition is met)
| DCR entity attributes | Mapping |
| --- | --- |
| vrStatus | "*CLOSED"* |
| vrStatusDetail | "*ACCEPTED*" |
| traceResponseReceivedDate | current time |
| oneKeyComment | *OK.responseComment* *\n* *ONEKEY ID = individualEidValidated or workplaceEidValidated* |
+ events are published to the $env-internal-onekey-dcr-change-events-in topic
Triggers
========
| Trigger action | Component | Action | Default time |
| --- | --- | --- | --- |
| **IN** Timer (cron) | dcr-service:TraceVRService | query mongo to get all SENT DCR's related to the PFORCERX process | every <T> hour |
| **OUT** Events | dcr-service:TraceVRService | generate the OneKeyDCREvent | every <T> hour |
Dependent components
====================
| Component | Usage |
| --- | --- |
| [DCR Service](https://confluence.pfizer.com/display/GMDM/DCR+Service) | Main component with flow implementation |
| [Hub Store](https://confluence.pfizer.com/display/GMDM/Hub+Store) | DCR and Entities Cache |
---
# OneKey: process DCR Change Events
**Page ID:** 209949303
**Page Link:** /display/GMDM/OneKey%3A+process+DCR+Change+Events
Description
===========
The process updates the DCR's based on the Change Request events received from `[ONEKEY|VOD]` (after trace VR method result). Based on the [IQVIA|VEEVA] Data Steward decision the `state`attribute contains relevant information to update DCR status. During this process also the comments created by IQVIA DS are retrieved and the relationship (optional step) between the DCR object and the newly created entity is created. DCR status is accepted only after the `[ONEKEY|VOD]` profile is created in Reltio, only then the Client will receive the ACCEPTED status. The process is checking Reltio with <T> delay and retries if the ETL load is still in progress waiting for `[ONEKEY|VOD]` profile.
Flow diagram
============
**OneKey variant**
![](/download/attachments/209949303/New%20DCR%20Process%20-%20OneKey%20process%20DCR%20Change%20Events%287%29.png?version=2&modificationDate=1721210852977&api=v2)
**Veeva variant:** ![](/download/attachments/209949303/image-2024-1-11_18-0-6.png?version=1&modificationDate=1714057114333&api=v2)
Steps
=====
* [OneKey: generate DCR Change Events (traceVR)](/pages/viewpage.action?pageId=209950500) publishes simple events to $env-internal-onekey-dcr-change-events-in: DCR\_CHANGED
+ `Veeva specific`: [Veeva: generate DCR Change Events (traceVR)](/pages/viewpage.action?pageId=379329922) publishes simple events to `$env-internal-veeva-dcr-change-events-in`: `DCR_CHANGED`
* Events are aggregated in a time window (recommended the window length 24 hours) and the last event is returned to the process after the window is closed.
* Events are processed in the Stream and based on the `OneKeyDCREvent.OneKeyChangeRequest``.vrStatus | VeevaDCREvent.VeevaChangeRequestDetails.vrStatus` attribute decision is made
* DCR is retrieved from the cache based on the `_id` of the DCR
* If the event state is **ACCEPTED**
+ Get Reltio entity `PfizerCustomerID` by `[ONEKEY|VOD]` crosswalk
+ If such crosswalk entity exists in Reltio:
- **PfizerGlobalCustomerId** is saved in Registry and will be returned to the Client
- During the process, the optional check is triggered - create the relation between the DCR object and newly created entities
* if DCRRegistry contain an empty list of `entityUris`, or some of the newly created entity is not present in the list, the Relation between this object and the DCR has to be created
+ **DCR**entity is updated in Reltio and the relation between the processed entity and the DCR entity
- Reltio source name (crosswalk. type): DCR
- Reltio relation type: `HCPtoDCR`or `HCOtoDCR`(depending on the object type)
+ Newly created entities uris should be retrieved by the `individualEidValidated` or `workplaceEidValidated` (it may be both) attributes from the events that represent the HCP or HCO crosswalks.
- The status in Reltio and in Mongo is updated
| DCR entity attributes | Mapping for OneKey | Mapping for Veeva |
| --- | --- | --- |
| VRStatus | CLOSED | |
| VRStatusDetail | state: ACCEPTED | |
| Comments | ONEKEY comments ({`VR.rsp.responseComments`}) ONEKEY ID = `i``ndividualEidValidated` or `workplaceEidValidated` | VEEVA comments = `VR.rsp.responseComments` VEEVA ID = `entityUris` |
| PfizerGlobalCustomerId | This is required in ACCEPTED status | |
+ If the `[ONEKEY|VOD]` does not exist in Reltio
- Regenerate the Event with a new timestamp to the input topic so this will be processed in the next <T> hours
- Update the Reltio DCR status
* | DCR entity attributes | Mapping |
| --- | --- |
| VRStatus | OPEN |
| VRStatusDetail | ACCEPTED |
- update the Mongo status to the `OK_NOT_FOUND | VEEVA_NOT_FOUND` and increase the "`retryCounter`" attribute
* If the event state is **REJECTED**
+ If a Reltio DS has already seen this request, REJECT the DCR and end the flow (if the initial target type is Reltio)
The status in Reltio and in Mongo is updated
| DCR entity attributes | Mapping |
| --- | --- |
| VRStatus | CLOSED |
| VRStatusDetail | state: REJECTED |
| Comments | `[ONEKEY|VOD]` comments ({`VR.rsp.responseComments`}) |
+ If this is based on the routing table and it was never sent to the Reltio DS, then create the DCR workflow and send this to the Reltio DS. Add the information comment that this was Rejected by the OneKey, so now Reltio DS has to decide if this should be REJECTED or APPLIED in Reltio. Add the comment that this is not possible to execute the sendTo3PartyValidation button in this case. Steps:
- Check if the initial target type is `[ONEKEY|VOD]`
- Use the DCR Request that was initially received from PforceRx and is a Domain Model request (after validation)
- Send the DCR to Reltio the service returns the following response:
* ACCEPTED (change request accepted by Reltio)
+ update the status to `DS_ACTION_REQUIERED`and in the comment add the following: "This DCR was REJECTED by the `[ONEKEY|VOD]` Data Steward with the following comment: <`[ONEKEY|VOD]` reject comment>. Please review this DCR in Reltio and APPLY or REJECT. It is not possible to execute the sendTo3PartyValidation button in this case"
+ initialize new Workflow in Reltio with the comment.
+ save data in the DCR entity status in Reltio and update Mongo DCR Registry with workflow ID and other attributes that were used in this Flow.
* REJECTED  (failure or error response from Reltio)
+ CLOSE the DCR with the information that DCR was REJECTED by the `[ONEKEY|VOD]` and Reltio also REJECTED the DCR. Add the error message from both systems in the comment.
Triggers
========
| Trigger action | Component | Action | Default time |
| --- | --- | --- | --- |
| **IN** Events incoming | dcr-service-2:DCROneKeyResponseStream dcr-service-2:DCRVeevaResponseStream (`$env-internal-veeva-dcr-change-events-in)` | process publisher full change request events in the stream | realtime: events stream processing |
Dependent components
====================
| Component | Usage |
| --- | --- |
| [DCR Service 2](https://confluence.pfizer.com/display/GMDM/DCR+Service+2) | Main component with flow implementation |
| [Manager](https://confluence.pfizer.com/display/GMDM/Manager) | Reltio Adapter  - API operations |
| [Publisher](https://confluence.pfizer.com/display/GMDM/Publisher) | Events publisher generates incoming events |
| [Hub Store](https://confluence.pfizer.com/display/GMDM/Hub+Store) | DCR and Entities Cache |
---
# Reltio: create DCR method - direct
**Page ID:** 209949292
**Page Link:** /display/GMDM/Reltio%3A+create+DCR+method+-+direct
Description
===========
Rest API method exposed in the [Manager](/display/GMDM/Manager) component responsible for submitting the Change Request to Reltio
Flow diagram
============
![](/download/attachments/209949292/New%20DCR%20Process%20-%20Reltio%20create%20DCR%20method%20-%20direct.png?version=1&modificationDate=1643129986367&api=v2)
Steps
=====
1. Receive the DCR request generated by DCR Service 2 component
2. Depending on the Action execute the method in the [Manager](/display/GMDM/Manager) component:
1. insert - Execute standard [Create/Update HCP/HCO/MCO](/pages/viewpage.action?pageId=164470018) operation with additional changeRequest.id parameter
2. update - Execute Update Attributes operation with additional [changeRequest.id](http://changeRequest.id) parameter
1. the combination of IGNORE\_ATTRIBUTE & INSERT\_ATTRIBUTE once updating existing parameter in Reltio
2. the INSERT\_ATTRIBUTE once adding new attribute to Reltio
3. delete - Execute Update Attribute operation with additional [changeRequest.id](http://changeRequest.id) parameter
1. the UPDATE\_END\_DATE on the entity to inactivate this profile
3. Based on the Reltio response the DCR Response is returned:
1. REQUEST\_ACCEPTED - Reltio processed the request successfully
2. REQUEST\_FAILED - Reltio returned the exception, Client will receive the detailed description in the errorMessage
Triggers
========
| Trigger action | Component | Action | Default time |
| --- | --- | --- | --- |
| REST call | DCR Service: POST /dcr2 | Create change Requests in Reltio | API synchronous requests - realtime |
Dependent components
====================
| Component | Usage |
| --- | --- |
| [DCR Service](https://confluence.pfizer.com/display/GMDM/DCR+Service) | Main component with flow implementation |
| [Hub Store](https://confluence.pfizer.com/display/GMDM/Hub+Store) | DCR and Entities Cache |
---
# Reltio: process DCR Change Events
**Page ID:** 209949300
**Page Link:** /display/GMDM/Reltio%3A+process+DCR+Change+Events
Description
===========
The process updates the DCR's based on the Change Request events received from Reltio(publishing). Based on the Data Steward decision the `state`attribute contains relevant information to update DCR status. During this process also the comments created by DS are retrieved and the relationship (optional step) between the DCR object and the newly created entity is created.
Flow diagram
============
![](/download/attachments/209949300/image-2023-11-20_15-5-57.png?version=1&modificationDate=1700489158010&api=v2)
Steps
=====
* Event publisher publishes simple events to `$env-internal-reltio-dcr-change-events-in`: DCR\_CHANGED("CHANGE\_REQUEST\_CHANGED") and DCR\_REMOVED("CHANGE\_REQUEST\_REMOVED")
* When the events do not contain the ThirdPartyValidation flag it means that DS APPLIED or REJECTED the DCR, the following logic is applied
+ Events are processed in the Stream and based on the `targetChangeRequest.state` attribute decision is made
- If the state is APPLIED or REJECTS, DCR is retrieved from the cache based on the `changeRequestURI`
* If DCR exists in Cache The status in Reltio is updated
| DCR entity attributes | Mapping |
| --- | --- |
| VRStatus | CLOSED |
| VRStatusDetail | state: APPLIED → ACCEPTED state: REJECTED → REJECTED |
* Otherwise, the events are rejected and the transaction is ended
- The `PfizerCustomerGlobalId` is retrieved for newly created entities in Reltio based on the main entity URI.
- During the process, the optional check is triggered - create the relation between the DCR object and newly created entities
* if `DCRRegistry` contain an empty list of `entityUris`, or some of the newly created entity is not present in the list, the Relation between this object and the DCR has to be created
+ **DCR**entity is updated in Reltio and the relation between the processed entity and the DCR entity
- Reltio source name (crosswalk. type): DCR
- Reltio relation type: `HCPtoDCR`or `HCOtoDCR`(depending on the object type)
- The comments added by the DataSteward during the processing of the Change request is retrieved using the following operation:
* ***GET*** /tasks?objectURI=entities/<id>
* The processInstanceComments is retrieved from the response and added to DCRRegistry.changeRequestComment
* Otherwise, when the events contain the ThirdPartyValidation flag it means that DS decided to send the DCR to IQVIA or VEEVA for the validation, the following logic is applied:
+ If the current targetType is ONEKEY | VEEVA
- REJECT the DCR and add the comment on the DCR in Retlio that "DCR was already processed by `[ONEKEY|VEEVA]` Data Stewards, REJECT because it is not allowed to send this DCR one more time to `[IQVIA|VEEVA]`"
+ If the current targetType is Reltio, it means that we can send this DCR to `[IQVIA|VEEVA]` for validation 
- Use the DCR Request that was initially received from PforceRx and is a Domain Model request (after validation)
- Execute the POST /dcr method in `[ONEKEY|VEEVA]` DCR Service, the service returns the following response:
* ACCEPTED - update the status to `[SENT_TO_OK|SENT_TO_VEEVA]`
* REJECTED - it means that some unexpected exception occurred in `[ONEKEY|VEEVA]`, or request was rejected by `[ONEKEY|VEEVA]`, or the ONEKEY crosswalk does not exist in Reltio, and `[ONEKEY|VEEVA]`service rejected this request
+ `Veeva specific`: When VOD crosswalk does not exist in Reltio, current version of profile is being sent to Veeva for validation independent from initial changes which where incorporated within DCR
Triggers
========
| Trigger action | Component | Action | Default time |
| --- | --- | --- | --- |
| **IN** Events incoming | dcr-service-2:DCRReltioResponseStream | process publisher full change request events in the stream | realtime: events stream processing |
Dependent components
====================
| Component | Usage |
| --- | --- |
| [DCR Service](https://confluence.pfizer.com/display/GMDM/DCR+Service) [DCR Service 2](https://confluence.pfizer.com/display/GMDM/DCR+Service+2) | Main component with flow implementation |
| [Manager](https://confluence.pfizer.com/display/GMDM/Manager) | Reltio Adapter  - API operations |
| [Publisher](https://confluence.pfizer.com/display/GMDM/Publisher) | Events publisher generates incoming events |
| [Hub Store](https://confluence.pfizer.com/display/GMDM/Hub+Store) | DCR and Entities Cache |
---
# Reltio: Profiles created by DCR
**Page ID:** 510266969
**Page Link:** /display/GMDM/Reltio%3A+Profiles+created+by+DCR
| DCR type | Approval/Reject | Record visibility in MDM | Crosswalk Type | Crosswalk Value | Source |
| --- | --- | --- | --- | --- | --- |
| DCR create for HCP/HCO | Approved by OneKey/VOD | HCP/HCO created in MDM | ONEKEY|VOD | onekey id | ONEKEY|VOD |
| Approved by DSR | HCP/HCO created in MDM | System source name from DCR (KOL\_OneView, PforceRx, etc) | DCR ID | System source name from DCR (KOL\_OneView, PforceRx, etc) |
| DCR edit for HCP/HCO | Approved by OneKey/VOD | HCP/HCO requested attribute updated in MDM | ONEKEY|VOD | | ONEKEY|VOD |
| Approved by DSR | HCP/HCO requested attribute updated in MDM | Reltio | entity uri | Reltio |
| DCR edit for HCPaddress/HCO address | Approved by OneKey/VOD | New address created in MDM, existing address marked as inactive | ONEKEY|VOD | | ONEKEY|VOD |
| Approved by DSR | New address created in MDM, existing address marked as inactive | Reltio | entity uri | Reltio |
---
# Veeva DCR flows
**Page ID:** 379332475
**Page Link:** /display/GMDM/Veeva+DCR+flows
Description
===========
The process is responsible for creating DCRs which are stored (Store VR) to be further transferred and processed by Veeva. Changes can be suggested by the DS using "Suggest" operation in Reltio and "Send to Third Party Validation" button. All DCRs are saved in the dedicated collection in HUB Mongo DB, required to gather metadata and trace the changes for each DCR request. During this process, the communication to Veeva Opendata is established via S3/SFTP communication. SubmitVR operation is executed to create a new ZIP files with DCR requests spread across multiple CSV files. The TraceVR operation is executed to check if Veeva responded to initial DCR Requests via ZIP file placed Inbound S3 dir.
The process is divided into 3 sections:
1. [Create DCR request - Veeva](/display/GMDM/Create+DCR+request+-+Veeva)
2. [Submit DCR Request - Veeva](/display/GMDM/Submit+DCR+Request+-+Veeva)
3. [Trace Validation Request - Veeva](/display/GMDM/Trace+Validation+Request+-+Veeva)
The below diagram presents an overview of the entire process. Detailed descriptions are available in the separated subpages.
Business process diagram for R1 phase
=====================================
![](/download/attachments/379332475/image-2024-4-18_14-34-45.png?version=1&modificationDate=1713443685880&api=v2)
Flow diagram
============
![](/download/attachments/379332475/image-2024-2-8_14-48-30.png?version=1&modificationDate=1707400110917&api=v2)
Steps
=====
* CreateVR
+ Process of saving DCR requests in Mongo Cache after being triggered by DCR Service 2.
+ DCR request information is translated to Veeva's model and stored in dedicated collection for Veeva DCRs.
* SubmitVR
+ The process of submitting VR stored in Mongo Cache to Veeva's SFTP via S3 bucket. The process aggregates events stored in Mongo Cache since last submit.
+ New ZIP is created with CSV files containing DCR request for Veeva. ZIP is placed in outbound dir in S3 bucket which is further synchronized to Veeva's SFTP.
+ Each DCR is updated with ZIP file name which was used to transfer request to Veeva.
* TraceVR
+ The process of tracing VR is triggered each <T> hours by Spring Scheduler.
+ Inbound S3 bucket is searched for ZIP files with CSVs containing DCR responses from Veeva. There are multiple dirs in S3 buckets, each for specific group of countries (currently CN and APAC).
+ Parts of DCR responses are spread across multiple files. Combined information is being processed.
+ Finally information about DCR is updated in Mongo Cache and events are produced to dedicated topic for DCR Service 2 for further processing.
Triggers
========
DCR service 2 is being triggered via `/dcr` API calls which are triggered by Data Stewards actions (R1 phase) → "Suggests 3rd party validation" which pushes DCR from Reltio to HUB.
Dependent components
====================
Described in the separated sub-pages for each process.
Design document for HUB development
===================================
1. Design → [VeevaOpenData-implementation.docx](/download/attachments/379129086/VeevaOpenData-implementation.docx?version=1&modificationDate=1701175388723&api=v2)
2. Reltio HUB-VOD mapping → [VeevaOpenDataAPACDataDictionary.xlsx](/download/attachments/379129086/VeevaOpenDataAPACDataDictionary.xlsx?version=1&modificationDate=1701175587157&api=v2)
3. VOD model description (v4) → [Veeva\_OpenData\_APAC\_Data\_Dictionary v4.xlsx](/download/attachments/379129086/Veeva_OpenData_APAC_Data_Dictionary%20v4.xlsx?version=1&modificationDate=1701337968690&api=v2)
---
# Create DCR request - Veeva
**Page ID:** 386814533
**Page Link:** /display/GMDM/Create+DCR+request+-+Veeva
Description
===========
The process of creating new DCR requests to the Veeva OpenData. During this process, new DCRs are created in DCRregistryVeeva mongo collection.
Flow diagram
============
![](/download/attachments/386814533/New%20DCR%20Process%20-%20Veeva%20create%20DCR%20method%20%28storeVR%29.png?version=1&modificationDate=1707223012200&api=v2)
Steps
=====
* Service is called by Rest API
* Input request is validated. If request is invalid - return response with status REJECTED
* Transform input request to Veeva DCR model
+ translate lookup codes to Veeva source codes
+ fill the Veeva DCR model with input request values
* Save DCR request to DCRRegistryVeeva mongo collection with status NEW
Mappings
========
DCR domain model→ VOD mapping file: [VeevaOpenDataAPACDataDictionary-mmor-mapping.xlsx](/download/attachments/379333348/VeevaOpenDataAPACDataDictionary-mmor-mapping.xlsx?version=1&modificationDate=1707149053810&api=v2)
Veeva integration guide
=======================
[![](/rest/documentConversion/latest/conversion/thumbnail/387176761/1)](/download/attachments/386814533/OpenData%20APAC%20-%20Integration%20Guide%20v8.pptx?version=1&modificationDate=1707228466760&api=v2)
---
# Submit DCR Request - Veeva
**Page ID:** 379333348
**Page Link:** /display/GMDM/Submit+DCR+Request+-+Veeva
Description
===========
The process of submitting new validation requests to the Veeva OpenData service via VeevaAdapter (communication with S3/SFTP) based on DCRRegistryVeeva mongo collection . During this process, new DCRs are created in VOD system.
Flow diagram
============
![](/download/attachments/379333348/New%20DCR%20Process%20-%20Veeva%20create%20DCR%20method%20%28submitVR%29.png?version=3&modificationDate=1707149327953&api=v2)
Steps
=====
### Veeva DCR service flow:
* Every N hours Veeva DCR requests with status NEW are queried in DCRRegistryVeeva store.
* DCR are group by country
* For each country:
+ - merge Veeva DCR requests - create one zip file for each country
- upload zip file to S3 location
- update DCR status to SENT if upload status is successful
| DCR entity attributes | Mapping |
| --- | --- |
| DCRID | Veeva VR Request Id |
| VRStatus | "OPEN" |
| VRStatusDetail | "SENT" |
| CreatedBy | MDM HUB |
| SentDate | current time |
### SFTP integration service flow:
* Every N  hours grab all zip files from S3 locations
* Upload files to corresponding SFTP server
Triggers
========
| Trigger action | Component | Action | Default time |
| --- | --- | --- | --- |
| **Spring scheduler** | mdm-veeva-dcr-service:VeevaDCRRequestSender | prepare ZIP files for VOD system | Called every specified interval |
Dependent components
====================
| Component | Usage |
| --- | --- |
| [Veeva adapter](/display/GMDM/Veeva+Adapter) | Upload DCR request to s3 location |
---
# Trace Validation Request - Veeva
**Page ID:** 379333358
**Page Link:** /display/GMDM/Trace+Validation+Request+-+Veeva
Description
===========
The process of tracing the VR changes based on the Veeva VR changes. During this process HUB, DCRRegistryVeeva Cache is triggered every <T> hour for SENT DCR's and check VR status using [Veeva Adapter](/display/GMDM/Veeva+Adapter) (s3/SFTP integration). After verification DCR event is sent to [DCR Service 2](/display/GMDM/DCR+Service+2)  Veeva response stream.
Flow diagram
============
*![](/download/attachments/379333358/Trace%20Veeva%20DCR%20-%20Veeva%20generate%20DCR%20Change%20Events%20%28traceVR.png?version=1&modificationDate=1707225871330&api=v2)*
Steps
=====
* Every N get all Veeva DCR responses using [Veeva Adapter](/display/GMDM/Veeva+Adapter)
* For each response:
+ check if status is terminal - (CHANGE\_ACCEPTED, CHANGE\_PARTIAL, CHANGE\_REJECTED, CHANGE\_CANCELLED)
- if not - go to next response
+ query DCRregistryVeeva mongo collection for DCR with given key and SENT status
+ get Veeva ID (vid\_\_v) from response file
+ generate Veeva DCR change event
+ update DCR status in DCRRegistryVeeva mongo collection
- resolution is CHANGE\_ACCEPTED, CHANGE\_PARTIAL
| DCR entity attributes | Mapping |
| --- | --- |
| VRStatus | "CLOSED" |
| VRStatusDetail | "ACCEPTED" |
| ResponseTime | veeva response completed date |
| Comments | veeva response resolution notes |
- resolution is CHANGE\_REJECTED, CHANGE\_CANCELLED
| DCR entity attributes | Mapping |
| --- | --- |
| VRStatus | "CLOSED" |
| VRStatusDetail | "REJECTED" |
| ResponseTime | veeva response completed date |
| Comments | veeva response resolution notes |
Triggers
========
| Trigger action | Component | Action | Default time |
| --- | --- | --- | --- |
| **IN** Spring scheduler | mdm-veeva-dcr-service:VeevaDCRRequestTrace | start trace validation request process | every <T> hour |
| **OUT** Kafka topic | mdm-dcr-service-2:VeevaResponseStream | update DCR status in Reltio, create relations | invokes Kafka producer for each veeva DCR response |
Dependent components
====================
| Component | Usage |
| --- | --- |
| [DCR Service 2](/display/GMDM/DCR+Service+2) | Process response event |
---
# Veeva: create DCR method (storeVR)
**Page ID:** 379332642
**Page Link:** /pages/viewpage.action?pageId=379332642
Description
===========
Rest API method exposed in the [Veeva DCR Service](/display/GMDM/Veeva+DCR+Service) component responsible for creating new DCR requests specific to Veeva OpenData (VOD) and storing them in dedicated collection for further submit. Since VOD enables communication only via S3/SFTP, it's required to use dedicated mechanism to actually trigger CSV/ZIP file creation and file placement in outbound directory. This will periodic call to Submit VR method will be scheduled once a day (with cron) which will in the end call VeevaAdapter with method createChangeRequest.
Flow diagram
============
![](/download/attachments/379332642/image-2023-12-27_16-19-37.png?version=1&modificationDate=1703690378393&api=v2)
Steps
=====
1. Receive the API request
2. Validate initial request
1. check if the Veeva crosswalk exists once there is an update on the profile
2. otherwise it's required to prepare DCR to create new Veeva profile
3. If there is any formal attribute missing or incorrect: skip request
3. Then the DCR is mapped to Veeva Request by invoking mapper between HUB DCR → VEEVA model
1. For mapping purpose below mapping table should be used
2. If there is not proper LOV mapping between HUB and Veeva, default fallback should be set to question mark → ?
4. Once proper request has been created,it should be stored as a `VeevaVRDetails` entry in dedicated `DCRRegistryVeeva` collection to be ready for actually send via Submit VR job and for future tracing purposes
5. Prepare return response for initial API request with below logic
1. Generate sample request after successful mongo insert →  `generateResponse(dcrRequest, RequestStatus.REQUEST_ACCEPTED, null, null)`
2. Generate error when validation or exception →  `generateResponse(dcrRequest, RequestStatus.REQUEST_FAILED, getErrorDetails(), null);`
Mapping HUB DCR → Veeva model
=============================
* Below table does not contain all new attributes which are new in Reltio. Only the most important ones were mentioned there.
* File [STTM Stats\_SG\_HK\_v3.xlsx](/download/attachments/379332642/STTM%20Stats_SG_HK_v3.xlsx?version=1&modificationDate=1703685207537&api=v2) contains full mapping requirements from Veeva OpenData to Reltio data model. It does contain full data mapping which should be covered in target DCR process for VOD.
| | | | | | | | | | | |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| **Reltio** | | **HUB** | | **VEEVA** | | | | | | |
| **Attribute Path** | **Details** | **DCR Request path** | **Details** | **File Name** | **Field Name** | **Required for Add Request?** | **Required for Change Request?** | **Description** | **Reference (RDM/LOV)** | NOTE |
| **HCO** | | | | | | | | | | |
| N/A | | Mongo Generated ID for this DCR | Kafka KEY | once mapping from HUB Domain DCRRequest take this from DCRRequestD.dcrRequestId: String, // HUB DCR request id - Mongo ID - required in ONEKEY service | change\_request | dcr\_key | Y | Y | Customer's internal identifier for this request | | |
| Change Requests comments | | extDCRComment | | change\_request | description | Y | Y | Requester free-text comments explaining the DCR | | |
| targetChangeRequest.createdBy | | createdBy | | change\_request | created\_by | Y | Y | For requestor identification | | |
| N/A | | if new objects - ADD, if veeva ID CHANGE | | change\_request | change\_request\_type | Y | Y | ADD\_REQUEST or CHANGE\_REQUEST | | |
| N/A | depends on suggested changes (check use-cases) | main entity object type HCP or HCO | | change\_request | entity\_type | Y | N | HCP or HCO | EntityType | |
| N/A | | Mongo Generated ID for this DCR | Kafka KEY | | change\_request\_hco | dcr\_key | Y | Y | Customer's internal identifier for this request | | |
| Reltio Uri and Reltio Type | when insert new profile | entities.HCO.updateCrosswalk.type (Reltio) entities.HCO.updateCrosswalk.value (Reltio id) and refId.entityURI | concatenate Reltio:rvu44dm | change\_request\_hco | entity\_key | Y | Y | Customer's internal HCO identifier | | |
| Crosswalks - VEEVA crosswalk | when update on VEEVA | entities.HCO.updateCrosswalk.type (VEEVA) entities.HCO.updateCrosswalk.value (VEEVA ID) | | change\_request\_hco | vid\_\_v | Y | N | Veeva ID of existing HCO to update; if blank, the request will be interpreted as an add request | | |
| configuration/entityTypes/HCO/attributes/OtherNames/attributes/Name | first element | TODO - add new attribute | | change\_request\_hco | alternate\_name\_1\_\_v | Y | N | | | |
| ?? | | ?? | | change\_request\_hco | business\_type\_\_v | Y | N | | HCOBusinessType | TO BE CONFIRMED |
| configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/FacilityType | | HCO.subTypeCode | | change\_request\_hcp | major\_class\_of\_trade\_\_v | N | N | | COTFacilityType | In PforceRx - Account Type, more info: [MR-9512](https://jira.pfizer.com/browse/MR-9512) - Getting issue details... STATUS |
| configuration/entityTypes/HCO/attributes/Name | | name | | change\_request\_hco | corporate\_name\_\_v | N | Y | | | |
| configuration/entityTypes/HCO/attributes/TotalLicenseBeds | | TODO - add new attribute | | change\_request\_hco | count\_beds\_\_v | N | Y | | | |
| configuration/entityTypes/HCO/attributes/Email/attributes/Email | email with rank 1 | emails | | change\_request\_hco | email\_1\_\_v | N | N | | | |
| configuration/entityTypes/HCO/attributes/Email/attributes/Email | email with rank 2 | | change\_request\_hco | email\_2\_\_v | N | N | | | |
| configuration/entityTypes/HCO/attributes/Phone/attributes/Number | phone type TEL.FAX with best rank | phones | | change\_request\_hco | fax\_1\_\_v | N | N | | | |
| configuration/entityTypes/HCO/attributes/Phone/attributes/Number | phone type TEL.FAX with worst rank | | change\_request\_hco | fax\_2\_\_v | N | N | | | |
| configuration/entityTypes/HCO/attributes/StatusDetail | | TODO - add new attribute | | change\_request\_hco | hco\_status\_\_v | N | N | | HCOStatus | |
| configuration/entityTypes/HCO/attributes/TypeCode | | typecode | | change\_request\_hco | hco\_type\_\_v | N | N | | HCOType | |
| configuration/entityTypes/HCO/attributes/Phone/attributes/Number | phone type TEL.OFFICE with best rank | phones | | change\_request\_hco | phone\_1\_\_v | N | N | | | |
| configuration/entityTypes/HCO/attributes/Phone/attributes/Number | phone type TEL.OFFICE with worst rank | | change\_request\_hco | phone\_2\_\_v | N | N | | | |
| configuration/entityTypes/HCO/attributes/Phone/attributes/Number | phone type TEL.OFFICE with worst rank | | change\_request\_hco | phone\_3\_\_v | N | N | | | |
| configuration/entityTypes/HCO/attributes/Country | | DCRRequest.country | | change\_request\_hco | primary\_country\_\_v | N | N | | | |
| configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty | elements from COT | specialties | | change\_request\_hco | specialty\_1\_\_v | N | N | | | |
| configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty | | change\_request\_hco | specialty\_10\_\_v | N | N | | Speciality | |
| configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty | | change\_request\_hco | specialty\_2\_\_v | N | N | | |
| configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty | | change\_request\_hco | specialty\_3\_\_v | N | N | | |
| configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty | | change\_request\_hco | specialty\_4\_\_v | N | N | | |
| configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty | | change\_request\_hco | specialty\_5\_\_v | N | N | | |
| configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty | | change\_request\_hco | specialty\_6\_\_v | N | N | | |
| configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty | | change\_request\_hco | specialty\_7\_\_v | N | N | | |
| configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty | | change\_request\_hco | specialty\_8\_\_v | N | N | | |
| configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty | | change\_request\_hco | specialty\_9\_\_v | N | N | | |
| configuration/entityTypes/HCO/attributes/Website/attributes/WebsiteURL | first element | websiteURL | | change\_request\_hco | URL\_1\_\_v | N | N | | | |
| configuration/entityTypes/HCO/attributes/Website/attributes/WebsiteURL | N/A | N/A | | change\_request\_hco | URL\_2\_\_v | N | N | | | |
| **HCP** | | | | | | | | | | |
| N/A | | Mongo Generated ID for this DCR | Kafka KEY | | change\_request\_hcp | dcr\_key | Y | Y | Customer's internal identifier for this request | | |
| Reltio Uri and Reltio Type | when insert new profile | entities.HCO.updateCrosswalk.type (Reltio) entities.HCO.updateCrosswalk.value (Reltio id) and refId.entityURI | concatenate Reltio:rvu44dm | change\_request\_hcp | entity\_key | Y | Y | Customer's internal HCP identifier | | |
| configuration/entityTypes/HCP/attributes/Country | | DCRRequest.country | | change\_request\_hcp | primary\_country\_\_v | Y | Y | | | |
| Crosswalks - VEEVA crosswalk | when update on VEEVA | entities.HCO.updateCrosswalk.type (VEEVA) entities.HCO.updateCrosswalk.value (VEEVA ID) | | change\_request\_hcp | vid\_\_v | N | Y | | | |
| configuration/entityTypes/HCP/attributes/FirstName | | firstName | | change\_request\_hcp | first\_name\_\_v | Y | N | | | |
| configuration/entityTypes/HCP/attributes/Middle | | middleName | | change\_request\_hcp | middle\_name\_\_v | N | N | | | |
| configuration/entityTypes/HCP/attributes/LastName | | lastName | | change\_request\_hcp | last\_name\_\_v | Y | N | | | |
| configuration/entityTypes/HCP/attributes/Nickname | | TODO - add new attribute | | change\_request\_hcp | nickname\_\_v | N | N | | | |
| configuration/entityTypes/HCP/attributes/Prefix | | prefix | | change\_request\_hcp | prefix\_\_v | N | N | | HCPPrefix | |
| configuration/entityTypes/HCP/attributes/SuffixName | | suffix | | change\_request\_hcp | suffix\_\_v | N | N | | | |
| configuration/entityTypes/HCP/attributes/Title | | title | | change\_request\_hcp | professional\_title\_\_v | N | N | | HCPProfessionalTitle | |
| configuration/entityTypes/HCP/attributes/SubTypeCode | | subTypeCode | | change\_request\_hcp | hcp\_type\_\_v | Y | N | | HCPType | |
| configuration/entityTypes/HCP/attributes/StatusDetail | | TODO - add new attribute | | change\_request\_hcp | hcp\_status\_\_v | N | N | | HCPStatus | |
| configuration/entityTypes/HCP/attributes/AlternateName/attributes/FirstName | | TODO - add new attribute | | change\_request\_hcp | alternate\_first\_name\_\_v | N | N | | | |
| configuration/entityTypes/HCP/attributes/AlternateName/attributes/LastName | | TODO - add new attribute | | change\_request\_hcp | alternate\_last\_name\_\_v | N | N | | | |
| configuration/entityTypes/HCP/attributes/AlternateName/attributes/MiddleName | | TODO - add new attribute | | change\_request\_hcp | alternate\_middle\_name\_\_v | N | N | | | |
| ?? | | TODO - add new attribute | | change\_request\_hcp | family\_full\_name\_\_v | N | N | | | TO BE CONFRIMED |
| configuration/entityTypes/HCP/attributes/DoB | | birthYear | | change\_request\_hcp | birth\_year\_\_v | N | N | | | |
| configuration/entityTypes/HCP/attributes/Credential/attributes/Credential | by rank 1 | TODO - add new attribute | | change\_request\_hcp | credentials\_1\_\_v | N | N | | | TO BE CONFIRMED |
| configuration/entityTypes/HCP/attributes/Credential/attributes/Credential | 2 | TODO - add new attribute | | change\_request\_hcp | credentials\_2\_\_v | N | N | | | In reltio there is attribute but not used |
| configuration/entityTypes/HCP/attributes/Credential/attributes/Credential | 3 | TODO - add new attribute | | change\_request\_hcp | credentials\_3\_\_v | N | N | | | "uri": "configuration/entityTypes/HCP/attributes/Credential/attributes/Credential", |
| configuration/entityTypes/HCP/attributes/Credential/attributes/Credential | 4 | TODO - add new attribute | | change\_request\_hcp | credentials\_4\_\_v | N | N | | | "lookupCode": "rdm/lookupTypes/Credential", |
| configuration/entityTypes/HCP/attributes/Credential/attributes/Credential | 5 | TODO - add new attribute | | change\_request\_hcp | credentials\_5\_\_v | N | N | | HCPCredentials | "skipInDataAccess": false |
| ?? | | TODO - add new attribute | | change\_request\_hcp | fellow\_\_v | N | N | | BooleanReference | TO BE CONFRIMED |
| configuration/entityTypes/HCP/attributes/Gender | | gender | | change\_request\_hcp | gender\_\_v | N | N | | HCPGender | |
| ?? Education ?? | | TODO - add new attribute | | change\_request\_hcp | education\_level\_\_v | N | N | | HCPEducationLevel | TO BE CONFRIMED |
| configuration/entityTypes/HCP/attributes/Education/attributes/SchoolName | | TODO - add new attribute | | change\_request\_hcp | grad\_school\_\_v | N | N | | | |
| configuration/entityTypes/HCP/attributes/Education/attributes/YearOfGraduation | | TODO - add new attribute | | change\_request\_hcp | grad\_year\_\_v | N | N | | | |
| ?? | | | | change\_request\_hcp | hcp\_focus\_area\_10\_\_v | N | N | | | TO BE CONFRIMED |
| ?? | | | | change\_request\_hcp | hcp\_focus\_area\_1\_\_v | N | N | | | |
| ?? | | | | change\_request\_hcp | hcp\_focus\_area\_2\_\_v | N | N | | | |
| ?? | | | | change\_request\_hcp | hcp\_focus\_area\_3\_\_v | N | N | | | |
| ?? | | | | change\_request\_hcp | hcp\_focus\_area\_4\_\_v | N | N | | | |
| ?? | | | | change\_request\_hcp | hcp\_focus\_area\_5\_\_v | N | N | | | |
| ?? | | | | change\_request\_hcp | hcp\_focus\_area\_6\_\_v | N | N | | | |
| ?? | | | | change\_request\_hcp | hcp\_focus\_area\_7\_\_v | N | N | | | |
| ?? | | | | change\_request\_hcp | hcp\_focus\_area\_8\_\_v | N | N | | | |
| ?? | | | | change\_request\_hcp | hcp\_focus\_area\_9\_\_v | N | N | | HCPFocusArea | |
| ?? | | | | change\_request\_hcp | medical\_degree\_1\_\_v | N | N | | | TO BE CONFRIMED |
| ?? | | | | change\_request\_hcp | medical\_degree\_2\_\_v | N | N | | HCPMedicalDegree | |
| configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialty | by rank from 1 to 100 | specialties | | change\_request\_hcp | specialty\_1\_\_v | Y | N | | | |
| configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialty | specialties | | change\_request\_hcp | specialty\_10\_\_v | N | N | | | |
| configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialty | specialties | | change\_request\_hcp | specialty\_2\_\_v | N | N | | | |
| configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialty | specialties | | change\_request\_hcp | specialty\_3\_\_v | N | N | | | |
| configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialty | specialties | | change\_request\_hcp | specialty\_4\_\_v | N | N | | | |
| configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialty | specialties | | change\_request\_hcp | specialty\_5\_\_v | N | N | | | |
| configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialty | specialties | | change\_request\_hcp | specialty\_6\_\_v | N | N | | | |
| configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialty | specialties | | change\_request\_hcp | specialty\_7\_\_v | N | N | | | |
| configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialty | specialties | | change\_request\_hcp | specialty\_8\_\_v | N | N | | | |
| configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialty | specialties | | change\_request\_hcp | specialty\_9\_\_v | N | N | | Specialty | |
| configuration/entityTypes/HCP/attributes/WebsiteURL | | TODO - add new attribute | | change\_request\_hcp | URL\_1\_\_v | N | N | | | |
| **ADDRESS** | | | | | | | | | | |
| | | Mongo Generated ID for this DCR | Kafka KEY | | change\_request\_address | dcr\_key | Y | Y | Customer's internal identifier for this request | | |
| Reltio Uri and Reltio Type | when insert new profile | entities.HCP OR HCO.updateCrosswalk.type (Reltio) entities.HCP OR HCO.updateCrosswalk.value (Reltio id) and refId.entityURI | concatenate Reltio:rvu44dm | change\_request\_address | entity\_key | Y | Y | Customer's internal HCO/HCP identifier | | |
| attributes/Addresses/attributes/PfizerAddressID | | address.refId | | change\_request\_address | address\_key | Y | Y | Customer's internal address identifier | | |
| attributes/Addresses/attributes/AddressLine1 | | addressLine1 | | change\_request\_address | address\_line\_1\_\_v | Y | N | | | |
| attributes/Addresses/attributes/AddressLine2 | | addressLine2 | | change\_request\_address | address\_line\_2\_\_v | N | N | | | |
| attributes/Addresses/attributes/AddressLine3 | | addressLine3 | | change\_request\_address | address\_line\_3\_\_v | N | N | | | |
| N/A | | N/A | A | change\_request\_address | address\_status\_\_v | N | N | | AddressStatus | |
| attributes/Addresses/attributes/AddressType | | addressType | | change\_request\_address | address\_type\_\_v | Y | N | | AddressType | |
| attributes/Addresses/attributes/StateProvince | | stateProvince | | change\_request\_address | administrative\_area\_\_v | Y | N | | AddressAdminArea | |
| attributes/Addresses/attributes/Country | | country | | change\_request\_address | country\_\_v | Y | N | | | |
| attributes/Addresses/attributes/City | | city | | change\_request\_address | locality\_\_v | Y | Y | | | |
| attributes/Addresses/attributes/Zip5 | | zip | | change\_request\_address | postal\_code\_\_v | Y | N | | | |
| attributes/Addresses/attributes/Source/attributes/SourceName attributes/Addresses/attributes/Source/attributes/SourceAddressID | when VEEVA map VEEVA ID to | sourceAddressId | | change\_request\_address | vid\_\_v | N | Y | | | |
| map from relationTypes/OtherHCOtoHCOAffiliations or relationTypes/ContactAffiliations | | This will be HCP.ContactAffiliation or HCO.OtherHcoToHCO affiliation | | | | | | | | |
| | | Mongo Generated ID for this DCR | Kafka KEY | | change\_request\_parenthco | dcr\_key | Y | Y | Customer's internal identifier for this request | | |
| | | HCO.otherHCOAffiliations.relationUri or HCP.contactAffiliations.relationUri | (from Domain model) information about Reltio Relation ID | change\_request\_parenthco | parenthco\_key | Y | Y | Customer's internal identifier for this relationship | RELATION ID | |
| | | KEY entity\_key from HCP or HCO (start object) | | change\_request\_parenthco | child\_entity\_key | Y | Y | Child Identifier in the HCO/HCP file | START OBJECT ID | |
| endObject entity uri mapped to refId.EntityURITargetObjectId | | KEY entity\_key from HCP or HCO (end object, by affiliation) | | change\_request\_parenthco | parent\_entity\_key | Y | Y | Parent identifier in the HCO file | END OBJECT ID | |
| | changes in Domain model mapping | map Reltion.Source.SourceName - VEEVA map Relation.Source.SourceValue - VEEVA ID | add to Domain model map if relation is from VEEVA ID | change\_request\_parenthco | vid\_\_v | N | Y | | | |
| | | start object entity type | | change\_request\_parenthco | entity\_type\_\_v | Y | N | | | |
| attributes/RelationType/attributes/PrimaryAffiliation | | if is primary TODO - add new attribute to otherHcoToHCO | | change\_request\_parenthco | is\_primary\_relationship\_\_v | N | N | | BooleanReference | |
| | | HCO\_HCO or HCP\_HCO | | change\_request\_parenthco | hierarchy\_type\_\_v | | | | RelationHierarchyType | |
| attributes/RelationType/attributes/RelationshipDescription | | type from affiliation based on ContactAffliation or OtherHCOToHCO affiliation | I think it will be 14-Emploted for HCP\_HCO and 4-Manages for HCO\_HCO but maybe we can map from affiliation.type | change\_request\_parenthco | relationship\_type\_\_v | Y | N | | RelationType | |
Mongo collection
================
All DCRs initiated by the `dcr-service-2` API and to be sent to Veeva will be stored in Mongo in new collection `DCRRegistryVeeva`. The idea is to gather all DCRs requested by the client through the day and schedule ‘`SubmitVR`’ process that will communicate with Veeva adapter.
Typical use case:
* Client requests 3 DCRs during the day
* `SubmitVR` contains the schedule that gathers all DCRs with **NEW** status created during the day and using VeevaAdapter to push requests to S3/SFTP.
Mapping Reltio canonical codes → Veeva source codes
===================================================
There are a couple of steps performed to find out a mapping for canonical code from Reltio to source code understood by VOD. Below steps are performed (in this order) once a code is found.
Veeva Defaults
--------------
Configuration is stored in `mdm-config-registry > config-hub/stage_apac/mdm-veeva-dcr-service/defaults`
The purpose of these logic is to select one of possible multiple source codes on VOD end for a single code on Pfizer side (1:N). The other scenario is when there is no actual source code for a canonical code on VOD end (1:0), however this is usually covered by fallback code logic.
There are a couple of files, each containing source codes for a specific attribute. The ones related to `HCO.Specialty` and `HCP.Specialty` have logic which selects proper code.
* Usually there are constructed as a three column CSV: `Country, Canonical Code, Source Code`
* For specific `Country` we're looking for `Canonical code` and then we're sending `Source code` as it is (no trim required)
+ Examples: `IN;SP.PD;PD` → `PD` source code will be sent to VOD
RDM lookups with RegExp
-----------------------
The main logic which is used to find out proper source code for canonical code. We're using codes configured in RDM, however mongo collection LookupValues are used. For specific canonical code (code) we looking for sourceMappings with source = VOD. Often country is embedded within source code so we're applying regexpConfig (more in Veeva Fallback section) to extract specific source code for particular country.
Veeva Fallback
--------------
Configuration is stored in `mdm-config-registry > config-hub/stage_apac/mdm-veeva-dcr-service/fallback`
* Available for a couple of attributes:
+ `hco-cot-facility-type.csv`
- COTFacilityType
+ `hco-specialty.csv`
- COTSpecialty
+ `hco-type-code.csv`
- HCOType
+ `hcp-specialty.csv`
- HCPSpecialty
+ `hcp-title.csv`
- HCPTitle
+ `hcp-type-code.csv`
- HCPSubTypeCode
* Usually files are constructed as a one column CSV, however the logic for extracting source code may be different
* Source code is extracted using RegExp for each parameter. Check `application.yml` for this mdm-veeva-dcr-server component - `mdm-inboud-services > mdm-veeva-dcr-service/src/main/resources/application.yml` to find out proper line and extract code sent to VOD.
+ Example value for `hco-specialty-type.csv: IN_?`
+ Regexp value for HCP.specialty: `regexpConfig > HCPSpecialty: ^COUNTRY_(.+)$`
+ Source code sent to VOD for India country: "`?`" (only question mark without country prefix)
Triggers
========
| Trigger action | Component | Action | Default time |
| --- | --- | --- | --- |
| REST call | [mdm-veeva-dcr-service](http://bitbucket-insightsnow.pfizer.com/projects/GMDM/repos/mdm-hub-inbound-services/browse/mdm-veeva-dcr-service): POST /dcr → veevaDCRService.createChangeRequest(request) | Creates DCR and stores it in collection without actual send to Veeva. | API synchronous requests - realtime |
Dependent components
====================
| Component | Usage |
| --- | --- |
| [DCR Service 2](/display/GMDM/DCR+Service+2) | Main component with flow implementation |
| [Hub Store](https://confluence.pfizer.com/display/GMDM/Hub+Store) | DCR and Entities Cache |
---
# Veeva: create DCR method (submitVR)
**Page ID:** 386796763
**Page Link:** /pages/viewpage.action?pageId=386796763
Description
===========
Gather all stored DCR entities in `DCRRegistryVeeva` collection (status = NEW) and sends them via S3/SFTP to Veeva OpenData (VOD). This method triggers CSV/ZIP file creation and file placement in outbound directory. This method is triggered from cron which invokes VeevaDCRRequestSender.sendDCRs() from the [Veeva DCR Service](/display/GMDM/Veeva+DCR+Service)
Flow diagram
============
![](/download/attachments/386796763/image-2023-12-28_13-33-44.png?version=1&modificationDate=1703766824597&api=v2)
Steps
=====
1. Receive the API request via scheduled trigger, usually every 24h (`senderConfiguration.schedulerConfig.fixedDelay`) at specific time of day (`senderConfiguration.schedulerConfig.initDelay)`
2. All DCR entities (`VeevaVRDetails`) with status NEW are being retrieved from `DCRRegistryVeeva` collection
3. Then `VeevaCreateChangeRequest` object is created which aggregates all CSV content which should be placed in actual CSV files.
1. Each object contains only DCRs specific for `country`
2. Each `country` has its own S3/SFTP directory structure as well as dedicated SFTP server instance
4. Once CSV files are created with header and content, they are packed into single ZIP file
5. Finally ZIP file is placed in outbound S3 directory
6. If file was placed
1. successfuly - then `VeevaChangeRequestACK` status = `SUCCESS`
2. otherwise - then `VeevaChangeRequestACK` status = `FAILURE` and process ends
7. Finally, status of `VeevaVRDetails` entity in `DCRRegistryVeeva` collection is updated and set to `SENT_TO_VEEVA`
Triggers
========
| Trigger action | Component | Action | Default time |
| --- | --- | --- | --- |
| Timer (cron) | [mdm-veeva-dcr-service](http://bitbucket-insightsnow.pfizer.com/projects/GMDM/repos/mdm-hub-inbound-services/browse/mdm-veeva-dcr-service): VeevaDCRRequestSender.sendDCRs() | Takes all unsent entities (status = NEW) from Veeva collection and actually puts file on S3/SFTP directory via `veevaAdapter.createDCRs` | Usually every 24h (`senderConfiguration.schedulerConfig.fixedDelay`) at specific time of day (`senderConfiguration.schedulerConfig.initDelay)` |
Dependent components
====================
| Component | Usage |
| --- | --- |
| [DCR Service 2](/display/GMDM/DCR+Service+2) | Main component with flow implementation |
| [Hub Store](https://confluence.pfizer.com/display/GMDM/Hub+Store) | DCR and Entities Cache |
---
# Veeva: generate DCR Change Events (traceVR)
**Page ID:** 379329922
**Page Link:** /pages/viewpage.action?pageId=379329922
Description
===========
The process is responsible for gathering DCR responses from Veeva OpenData (VOD). Responses are provided via CSV/ZIP files placed on S3/SFTP server in inbound directory which are specific for each country. During this process files should be retrieved, mapped from VOD to HUB DCR model and published to Kafka topic to be properly processed by DCR Service 2, [Veeva: process DCR Change Events](/display/GMDM/Veeva%3A+process+DCR+Change+Events).
Flow diagram
============
![](/download/attachments/379329922/image-2024-2-5_15-29-47.png?version=1&modificationDate=1707143387610&api=v2)
Source: [Lucid](https://lucid.app/lucidchart/4143a333-574e-4a47-b305-ce450202ce6a/edit?viewport_loc=-16%2C-268%2C3674%2C2038%2CwqrT~z3qGTzU&invitationId=inv_5f6a4de0-884a-4859-a13b-e9db321eb4b6)
Steps
=====
1. Method is trigger via cron, usually every 24h (`traceConfiguration.schedulerConfig.fixedDelay`) at specific time of day (`traceConfiguration.schedulerConfig.initDelay`)
2. For each country, each inbound directory in scanned for ZIP files
3. Each ZIP files (`<country>_DCR_Response_<Date>.zip`) should be unpacked and processed. A bunch of CSV files should be extracted. Specifically:
1. `change_request_response.csv` → it's a manifest file with general information in specific columns
1. `dcr_key` → ID of DCR which was established during DCR request creation
2. `entity_key` → ID of entity in Reltio, the same one we provided during DCR request creation
3. `entity_type` → type of entity (HCO, HCP) which is being modified via this DCR
4. `resolution` → has information whether DCR was accepted or rejected. Full list of values is below.
1. | **`resolution` value** | **Description** |
| --- | --- |
| CHANGE\_PENDING | This change is still processing and hasn't been resolved |
| CHANGE\_ACCEPTED | This change has been accepted without modification |
| CHANGE\_PARTIAL | This change has been accepted with additional changes made by the steward, or some parts of the change request have been rejected |
| CHANGE\_REJECTED | This change has been rejected in its entirety |
| CHANGE\_CANCELLED | This change has been cancelled |
5. `change_request_type`
1. | `change_request_type` value | Description |
| --- | --- |
| ADD\_REQUEST | whether DCR caused to create new profile in VOD with new `vid__v`  (Veeva id) |
| CHANGE\_REQUEST | just update of existing profile in VOD with existing and already known `vid__v` (Veeva id) |
2. `change_request_hcp_response.csv` - contains information about DCR related to HCP
3. `change_request_hco_response.csv` - contains information about DCR related to HCO
4. `change_request_address_response.csv` - contains information about DCR related to addresses which are related to specific HCP or HCO
5. `change_request_parenthco_response.csv` - contains information about DCR which correspond to relations between HCP and HCO, and HCO and HCO
6. File with log: `<country>_DCR_Request_Job_Log.csv` can be skipped. It does not contain any useful information to be processed automatically
4. For all DCR responses from VOD, we need to get corresponding DCR entity (`VeevaVRDetails)`from collection `DCRRegistryVeeva` should be selected.
5. In general, specific response files are not that important (VOD profiles updates will be ingested to HUB via ETL channel) however when new profiles are created (`change_request_response.csv.change_request_type = ADD_REQUEST`) we need to extract theirs Veeva ID.
1. We need to deep dive into `change_request_hcp_response.csv` or `change_request_hco_response.csv` to find `vid__v` (Veeva ID) for specific `dcr_key`
2. This new Veeva ID should be stored in `VeevaDCREvent.vrDetails.veevaHCPIds`
3. It should be further used as a crosswalk value in Reltio:
1. entities.HCO.updateCrosswalk.type (VEEVA)
2. entities.HCO.updateCrosswalk.value (VEEVA ID)
6. Once data has been properly mapped from Veeva to HUB DCR model, new `VeevaDCREvent` entity should be created and published to dedicated Kafka topic `$env-internal-veeva-dcr-change-events-in`
1. Please be advised, when the status of resolution is not final (CHANGE\_ACCEPTED, CHANGE\_REJECTED, CHANGE\_CANCELLED, CHANGE\_PARTIAL) we should not sent event to DCR-service-2
7. Then for each successfully processed DCR entity (`VeevaVRDetails`) in Mongo  DCRRegistryVeeva collection should be updated
1. | Veeva CSV: resolution | **Mongo: DCRRegistryVeeva** Entity: VeevaVRDetails.status: DCRRequestStatusDetails | Topic: $env-internal-veeva-dcr-change-events-in Event: VeevaDCREvent.vrDetails.vrStatus | Topic: $env-internal-veeva-dcr-change-events-in Event: VeevaDCREvent.vrDetails.vrStatusDetail |
| --- | --- | --- | --- |
| CHANGE\_PENDING | *status should not be updated at all (stays as SENT)* | *do not send events to DCR-service-2* | *do not send events to DCR-service-2* |
| CHANGE\_ACCEPTED | ACCEPTED | CLOSED | ACCEPTED |
| CHANGE\_PARTIAL | ACCEPTED | CLOSED | ACCEPTED *resolutionNotes / veevaComment should contain more information what was rejected by VEEVA DS* |
| CHANGE\_REJECTED | REJECTED | CLOSED | REJECTED |
| CHANGE\_CANCELLED | REJECTED | CLOSED | REJECTED |
8. Once files are processed, ZIP file should be moved from inbound to archive directory
Triggers
========
| Trigger action | Component | Action | Default time |
| --- | --- | --- | --- |
| **IN** Timer (cron) | [mdm-veeva-dcr-service](http://bitbucket-insightsnow.pfizer.com/projects/GMDM/repos/mdm-hub-inbound-services/browse/mdm-veeva-dcr-service): VeevaDCRRequestTrace.traceDCRs() | get DCR responses from S3/SFTP directory, extract CSV files from ZIP file and publish events to kafka topic | every <T> hour usually every 6h (`traceConfiguration.schedulerConfig.fixedDelay`) at specific time of day (`traceConfiguration.schedulerConfig.initDelay`) |
| **OUT** Events on Kafka Topic | [mdm-veeva-dcr-service](http://bitbucket-insightsnow.pfizer.com/projects/GMDM/repos/mdm-hub-inbound-services/browse/mdm-veeva-dcr-service): VeevaDCRRequestTrace.traceDCRs() $env-internal-veeva-dcr-change-events-in | VeevaDCREvent event published to topic to be consumed by DCR Service 2 | every <T> hour usually every 6h (`traceConfiguration.schedulerConfig.fixedDelay`) at specific time of day (`traceConfiguration.schedulerConfig.initDelay`) |
Dependent components
====================
| Component | Usage |
| --- | --- |
| [DCR Service 2](https://confluence.pfizer.com/display/GMDM/DCR+Service+2) | Main component with flow implementation |
| [Hub Store](https://confluence.pfizer.com/display/GMDM/Hub+Store) | DCR and Entities Cache |
---
Reference in New Issue View Git Blame Copy Permalink