anonymizer/data/HUB_DCR_output.md

DCR flows

Page ID: 415205424 Page Link: /display/GMDM/DCR+flows

Overview

DCR (Data Change Request) process helps to improve existing data in source systems. Proposal for change is being created by source systems a as DCR object (sometimes also called VR - Validation Request) which is usually being routed by MDM HUB to DS (Data Stewards) either in Reltio or in Third party validators (OneKey, Veeva OpenData). Response is provided twofold:

• response for specific DCR - metadata
• profile data update as a direct effect of a DCR processing - payload

General DCR process flow

High level solution architecture for DCR flow

Source: Lucid

Solution for OneKey (OK)

Solution for Veeva OpenData (VOD)

Architecture highlights

• Actors involved: PforceRX, Reltio, HUB, OneKey
• Key components: DCR Service 2 (second version) for AMER, EMEA, APAC, US tenants
• Process details:
  • DCRs are created directly by PforceRx using DCR's HUB API
  • PforceRx checks for DCR status updates every 24h → finds out which DCRs has been updated (since last check 24h ago) and the pulls details from each one with /dcr/_status
  • Integration with OneKey is realized by APIs - DCRs are created with /vr/submit and their status is verified every 8h with /vr/trace
  • Data profile updates (payload) are being delivered via CSV and S3 and ETLed (VOD batch) to Reltio with Deloitte's help
  • DCRRegistry & DCRRegistryVeeva collections are used in Mongo for tracking purposes

Architecture highlights

• Actors involved: Data Stewards in Reltio, HUB, Veeva OpenData (VOD)
• Key components: DCR Service 2 (second version) for AMER, EMEA, APAC, US tenants
• Process details:
  • DCRs are created by Data Stewards (DSRs) in Reltio via Suggest / Send to 3rd Party Validation - input for DSRs is being provided by reports from PforceRx
  • Communication with Veeva via S3<>SFTP and synchronization GMTF jobs. DCRs are sent and received in batches every 24h
  • DCRs metadata is being exchanged via multiple CSV files ZIPed
  • Data profile updates (payload) are being delivered via CSV and S3 and ETLed (VOD batch) to Reltio with Deloitte's help
  • DCRRegistry & DCRRegistryONEKEY collections are used in Mongofor tracking purposes

Solution for IQVIA Highlander (HL)

Solution for OneKey on GBLUS - sources ICEU, Engage, GRV

Architecture highlights

• Actors involved: Veeva on behalf of PforceRX, Reltio, HUB, IQVIA wrapper
• Key components: DCR Service (first version) for GBLUS tenant
• Process details:
  • DCRs are created by sending CSV requests by Veeva - based on information acquired from PforceRx
  • Integration HUB <> Veeva → via files and S3<>SFTP. HUB confirms DCR creation by returning file reports back to Veeva
  • Integration HUB <> IQVIA wrapper → via files and S3
  • HUB is responsible for translation of Veeva DCR CSV format to IQVIA CSV wrapper which then creates DCR in Reltio
  • Data Stewards approve or reject the DCRs in Reltio which updates data profiles accordingly.
  • PforceRx receives update about changes in Reltio
  • DCRRequest collection is used in Mongo for tracking purposes

Architecture highlights (draft)

• Actors involved: HUB, IQVIA wrapper
• Key components: DCR Service (first version) for GBLUS tenant
• Process details:
  • POST events from sources are captured - some of them are translated to direct DCRs, some of them are gathered and then pushed via flat files to be transformed into DCRs to OneKey

---

DCR generation process (China DCR)

Page ID: 164470008 Page Link: /pages/viewpage.action?pageId=164470008

The gateway supports following DCR types:

• NewHCP – created when new HCP is registered in Reltio and requires external validation
• NewHCOL1 – created when HCO Level 1 not found in Reltio
• NewHCOL2 – created when HCO Level 2 not found in Reltio
• MultiAffil – created when a profile has multiple affiliations

DCR generation processes are handled in two steps:

1. During HCP modification – if initial activation criteria are met, then a DCR request is generated and published to KAFKA -gw-dcr-requests topic.
2. In the next step, the internal Camel route DCRServiceRoute reads requests generated from the topic and processes as follows:
   1. checks if the time specified by delayPrcInSeconds elapsed since request generation – it makes sure that Reltio batch match process has finished and newly inserted profiles merge with the existing ones.
   2. checks if an entity, that caused DCR generation, still exists;
   3. checks full activation criteria (table below) on the latest state of the target entity, if criteria are not met then the request is closed
   4. creates DCR in Reltio
   5. updates external info
   6. creates PfizerDataChangeRequest entity in Reltio for tracking and exporting purposes.
3. Created DCRs are exported by the Informatica ETL process managed by IQIVIA
4. DCR applying process (reject/approve actions) are executed through MDM HUB DCR response API executed by the external app manged by MDE team.

The table below presents DCR activation criteria handled by system.

Table 9. DCR activation criteria
Rule	NewHCP	MultiAffiliation	NewHCOL2	NewHCOL1
Country in	CN	CN	CN	CN
Source in	GRV	GRV, MDE, FACE, EVR, CN3RDPARTY	GRV, FACE, CN3RDPARTY	GRV, FACE, CN3RDPARTY
ValidationStatus in	pending, partial-validated or, if merged: OV: notvalidated, GRV nonOV: pending/partial-validated	validated, pending	validated, pending	validated, pending
SpeakerStatus in	enabled, null	enabled, null	enabled, null	enabled, null
Workplaces count		>1
Hospital found	true	true	false	true
Department found	true	true		false
Similar DCR created in the past	false	false	false	false

Update: December 2021

• NewHCP DCR is now created if ValidationStatus is pending or partial-validated
• NewHCP DCR is also created if OV ValidationStatus is notvalidated, but most-recently updated GRV crosswalk provides non-ov ValidationStatus as pending or partial-validated - in case HCP gets merged into another entity upon creation/modification:
• DCR request processing history is now available in Kibana via Transaction Log - dashboard API Calls, transaction type "CreateDCRRoute"
• DCR response processing history (DCR approve/reject flow) is now available in Kibana via Transaction Log - dashboard API Calls, transaction type "DCRResponse"

HL DCR [Decommissioned April 2025]

Page ID: 164470085 Page Link: /pages/viewpage.action?pageId=164470085

Contacts

Vendor	Contact
PforceRX	DL-PForceRx-SUPPORT@pfizer.com
IQVIA (DCR Wrapper)	Pfizer-MDM-Support@iqvia.com

As a part of Highlander project, the DCR processing flow was created which realizes following scenarios:

1. Update HCP account details i.e. specialty, address, name (different sources of elements),
2. Add new HCP account with primary affiliation to an existing organization,
3. Add new HCP account with a new business account,
4. Update HCP and add affiliation to a new HCO,
5. Update HCP account details and remove existing details i.e. birth date, national id, …,
6. Update HCP account and add new non primary affiliation to an existing organization,
7. Update HCP account and add new primary affiliation to an existing organization,
8. Update HCP account inactivate primary affiliation. Person account has more than 1 affiliation,
9. Update HCP account inactivate non primary affiliation. Person account has more than 1 affiliation,
10. Inactivate HCP account,
11. Update HCP and add a private address,
12. Update HCP and update existing private address,
13. Update HCP and inactivate a private address,
14. Update HCO details i.e. address, name (different sources of elements),
15. Add new HCO account,
16. Update HCO and remove details,
17. Inactivate HCO account,
18. Update HCO address,
19. Update HCO and add new address,
20. Update HCO and inactivate address,
21. Update HCP's existing affiliation.

Above cases has been aggregated into six generic types in internal HUB model:

1. NEW_HCP_GENERIC - represents cases when the new HCP object is created with or without affiliation to HCO,
2. UPDATE_HCP_GENERIC - aggregates cases when the existing HCP object is changed,
3. DELETE_HCP_GENERIC - represents the case when HCP is deactivating,
4. NEW_HCO_GENERIC - aggregates scenarios when new HCO object is created with or without affiliations to parent HCO,
5. UPDATE_HCO_GENERIC - represents cases when existing HCO object is changing,
6. DELETE_HCO_GENERIC - represents the case when HCO is deactivating.

General Process Overview

Process steps:

1. Veeva uploads DCR request file to FTP location,
2. PforceRx Channel component downloads the DCR request file,
3. PforceRx Channel validates and maps each DCR requests to internal model,
4. PforceRx Channel sends the request to DCR Service,
5. DCR Service process the request: validating, enriching and mapping to Iqvia DCR Wrapper,
6. PforceRx Channel prepares the report file containing technical status of DCR processing - at this time, report will contain only requests which don't pass the validation,
7. Scheduled process in DCR Service, prepares the Wrapper requests file and uploads this to S3 location.
8. DCR Wrapper processes the file: creating DCRs in Reltio or rejecting the request due to errors. After that the response file is published to s3 location,
9. DCR Service downloads the response and updates DCRs status,
10. Scheduled process in PforceRx Channel gets DCR requests and prepares next technical report - at this time the report has technical status which comes from DCR Wrappper,
11. DCRs that was created by DCR Wrapper are reviewed by Data Stewards. DCR can be accepted or rejected,
12. After accepting or rejecting DCR, Reltio publishes the message about this event,
13. DCR Service consumes the message and updates DCR status,
14. PforceRx Channel gets DCR data to prepare a response file. The response file contains the final status of DCRs processing in Reltio.

Veeva DCR request file specification

The specification is available at following location:

https://pfizer-my.sharepoint.com/❌/r/personal/chinj2_pfizer_com/Documents/Mig%20In-Prog/Highlander/PMO/09%20Integration/LATAM%20Reltio%20DCR/DCR_Reltio_T144_Field_Mapping_Reltio.xlsx

DCR Wrapper request file specification

The specification is available at following link:

https://pfizer.sharepoint.com/❌/r/sites/HLDCR/Shared%20Documents/ReltioCloudMDM_LATAM_Highlander_DCR_DID_PFIZER__DEVMapping_v2.1.xlsx

OK DCR flows (GBLUS)

Page ID: 164469877 Page Link: /pages/viewpage.action?pageId=164469877

Description

The process is responsible for creating DCRs in Reltio and starting Change Requests Workflow for singleton entities created in Reltio. During this process, the communication to IQVIA OneKey VR API is established.  SubmitVR operation is executed to create a new Validation Request. The TraceVR operation is executed to check the status of the VR in OneKey. All DCRs are saved in the dedicated collection in HUB Mongo DB, required to gather metadata and trace the changes for each DCR request. Some changes can be suggested by the DS using "Suggest" operation in Reltio and "Send to Third Party Validation" button, the process "Data Steward OK Validation Request" is processing these changes and sends them to the OneKey service.

The process is divided into 4 sections:

1. Submit Validation Request
2. Trace Validation Request
3. Data Steward Response
4. Data Steward OK Validation Request

The below diagram presents an overview of the entire process. Detailed descriptions are available in the separated subpages.

Flow diagram

Model diagram

Steps

• SubmitVR
  • The process of submitting VR is triggered by the Reltio events. The process aggregates events in a time window and once the window is closed the processing is started.
  • During SubmitVR process checks are executed, getMatches operation in Relto is invoked to verify potential matches for the singleton entities.
  • Once all checks are correct new submitVR request is created in OneKey and DCR is saved in Reltio and in Mongo Cache.
• TraceVR
  • The process of tracing VR is triggered each  hours on Mongo DCR cache collection.
  • For each DCR the traceVR operation is executed in OneKey to verify the current status for the specific validation request.
  • Once the checks are correct the DCR is updated in Reltio and in Mongo Cache.
• Data Steward Response
  • The process is responsible for gathering changes on Change Requests objects from Reltio, the process is only accepting events without the ThirdPartyValidation flag
  • Based on the received change invoked by the Data Steward DCR is updated in Reltio and in Mongo Cache
• Data Steward OK Validation Request
  • The process is responsible for processing changes on Change Requests objects from Reltio, the process is only accepting events with the ThirdPartyValidation flag. This event is generated after DS clicks the "Send to Third Party Validation" button in Reltio.
  • The DS is "Suggesting" changes on the specified profile, these changes are next sent to HUB with the DCR event. The changes are not visible in Retlio, it is just a container that keeps the changes.
  • HUB is retrieving the "Preview" state from Reltio and calculating the changes that will send to OneKey WebService using submitVR operation
  • After successful submitVR response HUB is closing/rejecting the existing DCR in Reltio. The _reject operation has to be invoked on the current DCR in Reltio because the changes should no be applied to the profile. Changes are now validating in the OneKey system, and appropriate steps will be taken in the next phase (export changed data to Reltio or reject suggestion).

Triggers

Described in the separated sub-pages for each process.

Dependent components

Described in the separated sub-pages for each process.

Data Steward OK Validation Request

Page ID: 172306908 Page Link: /display/GMDM/Data+Steward+OK+Validation+Request

Description

The process the DS suggested changes based on the Change Request events received from Reltio(publishing) that are marked with the ThirdPartyValidation flag. The "suggested" changes are retrieved using the "preview" method and send to IQVIA OneKey or Veeva OpenData for validation. After successful submitVR response HUB is closing/rejecting the existing DCR in Reltio and additionally creates a new DCR object with relation to the entity in Reltio for tracking and status purposes. 

Because of the ONEKEY interface limitation, removal of attributes is send to IQVIA as a comment.

Flow diagram

Steps

• Event publisher publishes full enriched events to $env-internal-[onekeyvr|thirdparty]-ds-requests-in: DCR_CHANGED("CHANGE_REQUEST_CHANGED") and DCR_CREATED("CHANGE_REQUEST_CREATED")
• Only events with ExternalInfo and ThirdPartyValidation flag set to true and the Change Requests status equal to AWAITING_REVIEW are accepted in this process, otherwise, the event is rejected and processing ends.
• HUB DCR Cache is verified if any ReltioDCR requests exist and are not in a FAILED status, then processing goes to the next step.
• DCR request that contains targetChangeRequest is enriched with the current Entity data using HUB Cache
• Veeva specific: The entity is checked, If no VOD crosswalk exists, then "golden profile" parameters should be used with below logic
• The entity is checked, If active [ONEKEY|VOD] crosswalk exists the following steps are executed:
  • The suggested state of the entity is retrieved from Reltio using the getEntityWithChangeRequests operation (parameters - entityUri and the changeRequestId from the DCR event).
  • Current Entity and Preview Entity are compared using the following rules: (full attributes that are part of comparing process are described here)
    • Simple attributes (like FirstName/LastName):
      • Values are compared using the equals method. 
        • if differences are found the suggested value is taken.
        • If no differences are found
          • for mandatory, the current value is taken
          • for optional, the none value is taken (null)
    • Complex attributes (like Specialties/Addresses):
      • Whole nested attributes are matched using Reltio "uri" attributes key.
      • If there is a new Specialty/Address, the new suggested nested attribute is taken
        • Veeva specific: If there is a new Specialty/Addresses/Phone/Email/Medical degree*/HCP Focus area*, the new suggested nested attribute is taken. Since Veeva uses flat structure for these attributes, we need to calculate specialty attribute number (like specialty_5__v) to use when sending request. Attribute number = count of existing attributes +1.
      • If there is no new Specialty/Address and there is a change in the existing attribute, the suggested nested change is taken. If there are multiple suggested changes, the one with the highest Rank is taken.
      • If there are no changes
        • for mandatory, the current nested attribute that is connected with the ONEKEY crosswalk is taken.
        • for optional, the none nested attribute is taken (no need to send)
    • Contact Affiliations / OtherHCOtoHCOAffiliation:
      • If there are no changes, return current list
      • If there is new Contact Affiliation with ONEKEY crosswalk, add it to current list
    • Additional checks:
      • If there are changes associated with the other source (different than the [ONEKEY|VOD]), then these changes are ignored and the VR is saved in Reltio with comment listing what attributes were ignored e.g.: "Attributes: [YoB: 1956], [Email: engagetest123@test.com] ignored due to update on non-[onekey|VOD] attribute."
      • If attribute associated with [ONEKY|VOD] source is removed, a comment specifying what should be removed on [ONEKY|VOD] side is generated and sent to [ONEKY|VOD], e.g.: "Please remove attributes: [Address: 10648 Savannah Plantation Ct, 32832, Orlando, United States]."
  • DCRRequest object is created in Mongo for the flow state recording and generation of the new unique DCR ID for validation requests and data tracing.

    • DCR cache attributes	Values for IQVIA	Values for OK	Values for Veeva (R1)
      type	OK_VR	PFORCERX_DCR	RELTIO_SUGGEST
      status	DCRRequestStatusDetails (DCRRequestStatus.NEW, currentDate)
      createdBy	onekey-dcr-service	User which creates DCR via Suggest button in Reltio	User which creates DCR via Suggest button in Reltio
      date	now
      SendTo3PartyValidation	true (flag that indicates the DCR objects created by this process)

  • Calculated changes are mapped to the OneKey submitVR Request and it's submitted using API REST method POST /vr/submit.
    • Veeva specific:  submitting DCR request to Veeva requires creation of ZIPed CSV files with agreed structure and placed on S3 bucket
    • If the submission is successful then:
    • DCRRequest.status is updated to SENTwith [OK|VOD] request and response details
      • DCRentity is created in Reltio and the relation between the processed entity and the DCR entity
        • Reltio source name (crosswalk.type): DCR
        • Reltio relation type: HCPtoDCR or HCOtoDCR (depending on the object type)
        • DCR entity attributes:

          DCR entity attributes	Mapping for OneKey	Mapping for Veeva
          DCRID	OK VR Reqeust Id (cegedimRequestEid)	ID assigned by MDM HUB
          EntityURI	the processed entity URI
          VRStatus	"OPEN"
          VRStatusDetail	"SENT"
          Comments	optionally comments
          SentDate	current time
          SendTo3PartyValidation	true

    • Otherwise (FAILED)
    • DCRRequest.status is updated to FAILEDwith OK request and exception response details
    • DCRentity is created in Reltio and the relation between the processed entity and the DCR entity
      • Reltio source name (crosswalk.type): DCR
      • Reltio relation type: HCPtoDCRor HCOtoDCR(depending on the object type)
      • DCR entity attributes:

      • DCR entity attributes	Mapping
        DCRID	OK VR Reqeust Id (cegedimRequestEid)
        EntityURI	the processed entity URI
        VRStatus	"CLOSED"
        VRStatusDetail	"FAILED"
        Comments	ONEKEY service failed [exception details]
        SentDate	current time
        SendTo3PartyValidation	true

  • The current DCR object in Reltio is closed using the _reject operation - POST - /changeRequests/<id>/_reject
• Otherwise, If ONEKEY crosswalk does not exist, or the ONEKEY crosswalk is soft-deleted, or entity is EndDated: the following steps are executed:
  • DCRRequest object is created in Mongo for the flow state recording and generation of the new unique DCR ID for validation requests and data tracing.

    • DCR cache attributes	values
      type	DCRType.OK_VR
      status	DCRRequestStatusDetails (DCRRequestStatus.NEW, currentDate)
      created by	onekey-dcr-service
      date	now
      SendTo3PartyValidation	true (flag that indicates the DCR objects created by this process)

  • DCRRequest.status is updated to FAILEDand comment "No OK crosswalk available"
  • DCRentity is created in Reltio and the relation between the processed entity and the DCR entity
    • Reltio source name (crosswalk.type): DCR
    • Reltio relation type: HCPtoDCRor HCOtoDCR(depending on the object type)
    • DCR entity attributes:

    • DCR entity attributes	Mapping
      DCRID	OK VR Reqeust Id (cegedimRequestEid)
      EntityURI	the processed entity URI
      VRStatus	"CLOSED"
      VRStatusDetail	"REJECTED"
      Comments	No ONEKEY crosswalk available
      CreatedBy	MDM HUB
      SentDate	current time
      SendTo3PartyValidation	true

  • The current DCR object in Reltio is closed using the _reject operation - POST - /changeRequests//_reject
• END

ONEKEY Comparator (suggested changes)

HCP

Reltio Attribute	ONEKEY attribute	mandatory type	attribute type
FirstName	individual.firstName	optional	simple value
LastName	individual.lastName	mandatory	simple value
Country	isoCod2	mandatory	simple value
Gender	individual.genderCode	optional	simple lookup
Prefix	individual.prefixNameCode	optional	simple lookup
Title	individual.titleCode	optional	simple lookup
MiddleName	individual.middleName	optional	simple value
YoB	individual.birthYear	optional	simple value
Dob	individual.birthDay	optional	simple value
TypeCode	individual.typeCode	optional	simple lookup
PreferredLanguage	individual.languageEid	optional	simple value
WebsiteURL	individual.website	optional	simple value
Identifier value 1	individial.externalId1	optional	simple value
Identifier value 2	individial.externalId2	optional	simple value
Addresses[]	address.country address.city address.addressLine1 address.addressLine2 address.Zip5	mandatory	complex (nested)
Specialities[]	individual.speciality1 / 2 / 3	optional	complex (nested)
Phone[]	individual.phone	optional	complex (nested)
Email[]	individual.email	optional	complex (nested)
Contact Affiliations[]	workplace.usualName workplace.officialName workplace.workplaceEid	optional	Contact Affiliation
ONEKEY crosswalk	individual.individualEid	mandatory	ID

HCO

Reltio Attribute	ONEKEY attribute	mandatory type	attribute type
Name	workplace.usualName workplace.officialName	optional	simple value
Country	isoCod2	mandatory	simple value
OtherNames.Name	workplace.usualName2	optional	complex (nested)
TypeCode	workplace.typeCode	optional	simple lookup
WebisteWebsiteURL	workplace.website	optional	complex (nested)
Addresses[]	address.country address.city address.addressLine1 address.addressLine2 address.Zip5	mandatory	complex (nested)
Specialities[]	workplace.speciality1 / 2 / 3	optional	complex (nested)
Phone[] (!FAX)	workplace.telephone	optional	complex (nested)
Phone[] (FAX)	workplace.fax	optional	complex (nested)
Email[]	workplace.email	optional	complex (nested)
ONEKEY crosswalk	workplace.workplaceEid	mandatory	ID

Triggers

Trigger action	Component	Action	Default time
IN Events incoming	mdm-onekey-dcr-service:ChangeRequestStream	process publisher full change request events in the stream that contain ThirdPartyValidation flag	realtime: events stream processing

Dependent components

Component	Usage
OK DCR Service	Main component with flow implementation
Veeva DCR Service	Main component with flow implementation
Publisher	Events publisher generates incoming events
Hub Store	DCR and Entities Cache

---

Data Steward Response

Page ID: 164469841 Page Link: /display/GMDM/Data+Steward+Response

Description

The process updates the DCR's based on the Change Request events received from Reltio(publishing). Based on the Data Steward decision the state attribute contains relevant information to update DCR status.

Flow diagram

Steps

• Event publisher publishes simple events to $env-internal-[onekeyvr|veeva]-change-requests-in: DCR_CHANGED("CHANGE_REQUEST_CHANGED") and DCR_REMOVED("CHANGE_REQUEST_REMOVED")
• Only the events without the ThirdPartyValidation flag are accepted, otherwise, the event is Rejected and the process is ended.
• Events are processed in the Stream and based on the targetChangeRequest.state attribute decision is made
  • If the state is APPLIED or REJECTS, DCR is retrieved from the cache based on the changeRequestURI
    • If DCR exists in Cache The status in Reltio is updated

      DCR entity attributes	Mapping
      VRStatus	CLOSED
      VRStatusDetail	state: APPLIED → ACCEPTED state: REJECTED → REJECTED

    • Otherwise, the events are rejected and the transaction is ended
  • Otherwise, the events are rejected and the transition is ended.

Triggers

Trigger action	Component	Action	Default time
IN Events incoming	mdm-onekey-dcr-service:OneKeyResponseStream mdm-veeva-dcr-service:veevaResponseStream	process publisher full change request events in stream	realtime: events stream processing

Dependent components

Component	Usage
OK DCR Service	Main component with flow implementation
Veeva DCR Service	Main component with flow implementation
Publisher	Events publisher generates incoming events
Hub Store	DCR and Entities Cache

---

Submit Validation Request

Page ID: 164469875 Page Link: /display/GMDM/Submit+Validation+Request

Description

The process of submitting new validation requests to the OneKey service based on the Reltio change events aggregated in time windows. During this process, new DCRs are created in Reltio.

Flow diagram

Steps

• Event publisher publishes simple events to $env-internal-onekeyvr-in including HCP_*, HCO_*, ENTITY_MATCHES_CHANGED
• Events are aggregated in a time window (recommended the window length 4 hours) and the last event is returned to the process after the window is closed.
• Simple events are enriched with the Entity data using HUB Cache
• Then, the following checks are executed
  • check if at least one crosswalk create date is equal or above for a given source name and cut off date specified in configuration - section submitVR/crosswalkDecisionTables
  • check if entity attribute values match specified in configuration
  • check if there is no valid DCR created for the entity
  • check if the entity is active
  • check if the OK crosswalk doesn't exist after the full entity retrieval from the HUB cache
  • match category is not 99
  • GetMatches operation from Reltio returns 0 potential matches
• If any check is negative then the process is aborted.
• DCRRequest object is created in Mongo for the flow state recording and generation of the new unique DCR ID for validation request and data tracing.
• The entity is mapped to OK VR Request and it's submitted using API REST method POST /vr/submit.
• If the submission is successful then:
  • DCRRequest.status is updated to SENT with OK request and response details
  • DCR entity is created in Reltio and the relation between the processed entity and the DCR entity
    • Reltio source name (crosswalk.type): DCR
    • Reltio relation type: HCPtoDCRor HCOtoDCR(depending on the object type)
    • DCR entity attributes:

      DCR entity attributes	Mapping
      DCRID	OK VR Reqeust Id (cegedimRequestEid)
      EntityURI	the processed entity URI
      VRStatus"	"OPEN"
      VRStatusDetail	"SENT"
      CreatedBy	MDM HUB
      SentDate	current time

• Otherwise FAILEDstatus is recorded in DCRRequestwith an OK error response.
  • DCRRequest.status is updated to FAILED with OK request and exception response details
  • DCR entity is created in Reltio and the relation between the processed entity and the DCR entity
    • Reltio source name (crosswalk.type): DCR
    • Reltio relation type: HCPtoDCR or HCOtoDCR (depending on the object type)
    • DCR entity attributes:

      DCR entity attributes	Mapping
      DCRID	OK VR Reqeust Id (cegedimRequestEid)
      EntityURI	the processed entity URI
      VRStatus	"CLOSED"
      VRStatusDetail	"FAILED"
      Comments	ONEKEY service failed [exception details]
      CreatedBy	MDM HUB
      SentDate	current time

Triggers

Trigger action	Component	Action	Default time
IN Events incoming	mdm-onekey-dcr-service:OneKeyStream	process publisher simple events in stream	events stream processing with 4h time window events aggregation
OUT API request	one-key-client:OneKeyIntegrationService.submitValidation	submit VR request to OneKey	invokes API request for each accepted event

Dependent components

Component	Usage
OK DCR Service	Main component with flow implementation
Publisher	Events publisher generates incoming events
Manager	Reltio Adapter for getMatches and created operations
OneKey Adapter	Submits Validation Request
Hub Store	DCR and Entities Cache

Mappings

Reltio → OK mapping file: onkey_mappings.xlsx

OK mandatory / required fields: VR - Business Fields Requirements(Pfizer).xlsx

OneKey Documentation

Trace Validation Request

Page ID: 164469983 Page Link: /display/GMDM/Trace+Validation+Request

Description

The process of tracing the VR changes based on the OneKey VR changes. During this process HUB, DCR Cache is triggered every hour for SENT DCR's and check VR status using OneKey web service. After verification DCR is updated in Reltio or a new Workflow is started in Reltio for the Data Steward manual validation.

Flow diagram

Steps

• Every N  hours OK VR requests with status SENT are queried in DCRRequests store*.*
• For each open requests, its status is checked it OK using REST API method /vr/trace
• The first check is the VR.rsp.statusattribute, checking if the status is SUCCESS
• Next, if the process status (VR.rsp.results.processStatus) is REQUEST_PENDING_OKE | REQUEST_PENDING_JMS | REQUEST_PROCESSED or OK data export date (VR.rsp.results.trace6CegedimOkcExportDate) is earlier than 24 hours then the processing of the request is postponed to the next check
  • exportDate or processStatusare optional and can be null.
  • The process goes to the next step only if processStatus is  REQUEST_RESPONDED | RESPONSE_SENT
  • The process is blocked to next check only if  trace6CegedimOkcExportDateis not null and is earlier than 24h
• If the processStatusis validated and VR.rsp.results*.responseStatus*is VAS_NOT_FOUND | VAS_INCOHERENT_REQUEST | VAS_DUPLICATE_PROCESS then DCR is being closed with status REJECTED

  DCR entity attributes	Mapping
  VRStatus"	"CLOSED"
  VRStatusDetail	"REJECTED"
  ReceivedDate	current time
  Comments	OK.responseComment

• Before these 2 next steps, the current Entity status is retrieved from HUB Cache. This is required to check if the entity was merged with OK entity.
  • if responseStatus isVAS_FOUND | VAS_FOUND_BUT_INVALID and OK crosswalk exists in Reltio entity which value equals to OK validated id (individualEidValidated or workplaceEidValidated) then DCR is closed with status ACCEPTED.

    DCR entity attributes	Mapping
    VRStatus"	"CLOSED"
    VRStatusDetail	"ACCEPTED"
    ReceivedDate	current time
    Comments	OK.responseComment

  • if responseStatus isVAS_FOUND | VAS_FOUND_BUT_INVALID butOK crosswalk doesn't exist in Reltio then Relio DCR Request is created and workflow task is triggered for Data Steward review. DCR status entity is updated with DS_ACTION_REQUIRED status.

    DCR entity attributes	Mapping
    VRStatus"	"OPEN"
    VRStatusDetail	"DS_ACTION_REQUIRED "
    ReceivedDate	current time
    Comments	OK.responseComment

  • GET /changeRequests operation is invoked to get a new change request ID and start a new workflow
  • POST /workflow/_initiate operation is invoked to init new Workflow in Reltio

    Workflow attributes	Mapping
    changeRequest.uri	ChangeRequest Reltio URI
    changeRequest.changes	Entity URI
    comment	individualEidValidated or workplaceEidValidated

  • POST /entities?changeRequestId= - operation is invoked to update change request Entity container with DCR Status to Closed, this change is only visible in Reltio once DS accepts the DCR. 

    Body attributes	Mapping
    attributes	"DCRRequests": [ { "value": { "VRStatus": [ { "value": "CLOSED" } ] }, "refEntity": { "crosswalks": [ { "type": "configuration/sources/DCR", "value": "$requestId", "dataProvider": false, "contributorProvider": true }, { "type": "configuration/sources/DCR", "value": "$requestId_REF", "dataProvider": true, "contributorProvider": false } ] }, "refRelation": { "crosswalks": [ { "type": "configuration/sources/DCR", "value": "$requestId_REF" } ] } } ]
    crosswalks	"crosswalks": [ { "type": "configuration/sources/<source crosswalk>", "value": "<source value>", "dataProvider": false, "contributorProvider": true, "deleteDate": "" }, { "type": "configuration/sources/DCR", "value": "$requestId_CR", "dataProvider": true, "contributorProvider": false, "deleteDate": "" } ]

Triggers

Trigger action	Component	Action	Default time
IN Timer (cron)	mdm-onekey-dcr-service:TraceVRService	query mongo to get all SENT DCR's related to OK_VR process	every hour
OUT API request	one-key-client:OneKeyIntegrationService.traceValidation	trace VR request to OneKey	invokes API request for each DCR

Dependent components

Component	Usage
OK DCR Service	Main component with flow implementation
Manager	Reltio Adapter for GET /changeRequests and POST /workflow/_initiate operations
OneKey Adapter	TraceValidation Request
Hub Store	DCR and Entities Cache

---

PforceRx DCR flows

Page ID: 209949183 Page Link: /display/GMDM/PforceRx+DCR+flows

Description

MDM HUB exposes Rest API to create and check the status of DCR. The process is responsible for creating DCRs in Reltio and starting Change Requests Workflow DCRs created in Reltio or creating the DCRs (submitVR operation) in ONEKEY. DCR requests can be routed to an external MDM HUB instance handling the requested country. The action is transparent to the caller. During this process, the communication to IQVIA OneKey VR API / Reltio API is established. The routing decision depends on the market, operation type, or changed profile attributes.

Reltio API:  createEntity (with ChangeReqest) operation is executed to create a completely new entity in the new Change Request in Reltio. attributesUpdate (with ChageRequest) operation is executed after calculation of the specific changes on complex or simple attributes on existing entity - this also creates a new Change Request.  Start Workflow operation is requested at the end, this starts the Wrofklow for the DCR in Reltio so the change requests are started in the Reltio Inbox for Data Steward review.

IQVIA API: SubmitVR operation is executed to create a new Validation Request. The TraceVR operation is executed to check the status of the VR in OneKey.

All DCRs are saved in the dedicated collection in HUB Mongo DB, required to gather metadata and trace the changes for each DCR request. The DCR statuses are updated by consuming events generated by Reltio or periodic query action of open DCRs in OneKey

The Data Steward can decide to route a DCR to IQVIA as well - some changes can be suggested by the DS using the "Suggest" operation in Reltio and "Send to Third Party Validation" button, the process "Data Steward OK Validation Request" is processing these changes and sends them to the OneKey service.

The below diagram presents an overview of the entire process. Detailed descriptions are available in the separated subpages.

API doc URL: https://api-emea-nprod-gbl-mdm-hub.pfizer.com/api-dcr-spec-emea-dev/swagger-ui/index.html

Flow diagram

DCR Service High-Level Architecture

DCR HUB Logical Architecture

Model diagram

Flows:

• Create DCR
  • The client call API Post /dcr method and pass the request in JSON format to MDM HUB DCR service
  • The request is validated against the following rules:
    • mandatory fields are set
    • reference object HCP,HCO are available in Reltio
    • referenced attributes like specialties, addresses are in the changed object
  • The service evaluates the target system based on country, operation type (create, update), changed attributes. The process is controlled by the decision table stored in the config.
  • The DCR is created in the target system through the API
  • The result is stored in the registry. DCR information entity is created in Reltio for tracking.
  • The status with created DCR object ids are returned in response to the Client
• Get DCR status
  • The client calls GET /dcr/_status method
  • The DCR service queries DCR registry in Mongo and returns the status to the Client.
  • There are processes updating dcr status in the registry:
    • DCR change events are generated by Reltio when DCR is accepted or rejected by DS. Events are processed by the service.
• Reltio: process DCR Change Events
  • DCR change events are generated by Reltio when DCR is accepted or rejected by DS. Events are processed by the service.
• OneKey: process DCR Change Events
  • DCR change events are generated by the OneKey service when DCR is accepted or rejected by DS. Events are processed by the service.
• OneKey: generate DCR Change Events (traceVR)
  • Every x configured hours the OneKey status method is queried to get status for open validation requests.
• Reltio: create DCR method - direct
  • direct API method that creates DCR in Reltio (contains mapping and logic description)
• OneKey: create DCR method (submitVR) - direct
  • direct API method that creates DCR in OneKey - executes the submitVR operation (contains mapping and logic description)

Triggers

Described in the separated sub-pages for each process.

Dependent components

Described in the separated sub-pages for each process.

Create DCR

Page ID: 209949185 Page Link: /display/GMDM/Create+DCR

Description

The process creates change requests received from PforceRx Client and sends the DCR to the specified target service - Reltio, OneKey or Veeva OpenData (VOD). DCR is created in the system and then processed by the data stewards. The status is asynchronously updated by the HUB processes, Client represents the DCR using a unique extDCRRequestId value. Using this value Client can check the status of the DCR (Get DCR status).

Flow diagram

Source: Lucid

Source: Lucid

DCR Service component perspective

Steps

1. Clients execute the API POST /dcr request
2. Kong receives requests and handles authentication
3. If the authentication succeeds the request is forwarded to the dcr-service-2 component,
4. DCR Service checks permissions to call this operation and the correctness of the request, then the flow is started and the following steps are executed:
   1. Parse and validate the dcr request. The validation logic checks the following:
      1. Check if the list of DCRRequests contains unique extDCRRequestId.
         1. Requests that are duplicate will be rejected with the error message - "Found duplicated request(s)"
      2. For each DCRRequest in the input list execute the following checks:
         1. Users can define the following number of entities in the Request:
            1. at least one entity has to be defined, otherwise, the request will be rejected with an error message - "No entities found in the request"
            2. single HCP
            3. singe HCO
            4. singe HCP with single HCO
            5. two HCOs
         2. Check if the main reference objects exist in Reltio for update and delete action
            1. HCP.refId or HCO.refId, user have to specify one of:
               1. CrosswalkTargetObjectId - then the entity is retrieved from Reltio using get entity by crosswalk operation
               2. EntityURITargetObjectId - then the entity is retrieved from Reltio using get entity by uri operation
               3. PfizerCustomerIdTargetObjectId - then the entity is retrieved from Reltio using search operation by the PfizerGlobalCustomerID
         3. Attributes validation:
            1. Simple attributes - like firstName/lastName e.t.c
               1. for update action on the main object:
                  1. if the input parameter is defined with an empty value - "" - this will result in the removal of the target attribute
                  2. if the input parameter is defined with a non-empty value - this will result in the update of the target attribute
            2. Nested attributes - like Specialties/Addresses e.t.c
               1. for each attribute, the user has to define the refId to uniquely identify the attribute
                  1. For action "update" - if the refId is not found in the target object request will be rejected with a detailed error message
                  2. For action "insert" - the refId is not required - new reference attribute will be added to the target object
         4. Changes validation:
            1. If the validation detected 0 changes (during comparison of applying changes and the target entity) -  the request is rejected with an error message - "No changes detected"
   2. Evaluate dcr service (based on the decision table config)
      1. The following decision table is defined to choose the target service
         1. LIST OF the following combination of attributes:

            attribute	description
            userName	the user name that executes the request
            sourceName	the source name of the Main object
            country	the county defined in the request
            operationType	the operation type for the Main object { insert, update, delete }
            affectedAttributes	the list of attributes that the user is changing
            affectedObjects	{ HCP, HCO, HCP_HCO }
            	RESULT →  TargetType {Reltio, OneKey, Veeva}

         2. Each attribute in the configuration is optional.
         3. The decision table is making the validation based on the input request and the main object- the main object is HCP, if the HCP is empty then the decision table is checking HCO.
         4. The result of the decision table is the TargetType, the routing to the Reltio MDM system, OneKey or Veeva service.
   3. Execute target service (reltio/onekey/veeva)
      1. Reltio: create DCR method - direct
      2. OneKey: create DCR method (submitVR) - direct
      3. Veeva: create DCR method (storeVR)
   4. Create DCR in Reltio and save DCR in DCR Registry
      • If the submission is successful then:
        • DCRentity is created in Reltio and the relation between the processed entity and the DCR entity
          • Reltio source name (crosswalk.type): DCR
          • Reltio relation type: HCPtoDCRor HCOtoDCR(depending on the object type)
            • for "create" and "delete" operation the Relation have to be created between objects
            • if this is just the "insert" operation the Relation will be created after the acceptance of the Change Request in Reltio - Reltio: process DCR Change Events
          • DCR entity attributes once sent to OneKey

            DCR entity attributes	Mapping
            DCRID	extDCRRequestId
            EntityURI	the processed entity URI
            VRStatus	"OPEN"
            VRStatusDetail	"SENT_TO_OK"
            CreatedBy	MDM HUB
            SentDate	current time
            CreateDate	current time
            CloseDate	if REJECTED
            dcrType	evaluate based on config: dcrTypeRules: - type: CR0 size: 1 action: insert entity: com.pfizer.mdm.api.dcr2.HCP

          • DCR entity attributes once sent to Veeva

            DCR entity attributes	Mapping
            DCRID	extDCRRequestId
            EntityURI	the processed entity URI
            VRStatus	"OPEN"
            VRStatusDetail	"SENT_TO_VEEVA"
            CreatedBy	MDM HUB
            SentDate	current time
            CreateDate	current time
            CloseDate	*if REJECTED
            dcrType	evaluate based on config: dcrTypeRules: - type: CR0 size: 1 action: insert entity: com.pfizer.mdm.api.dcr2.HCP

          • DCR entity attributes once sent to Reltio → action is passed to DS and workflow is started.

            DCR entity attributes	Mapping
            DCRID	extDCRRequestId
            EntityURI	the processed entity URI
            VRStatus	"OPEN"
            VRStatusDetail	"DS_ACTION_REQUIRED "
            CreatedBy	MDM HUB
            SentDate	current time
            CreateDate	current time
            CloseDate	if REJECTED
            dcrType	evaluate based on config: dcrTypeRules: - type: CR0 size: 1 action: insert entity: com.pfizer.mdm.api.dcr2.HCP

        • Mongo Update: DCRRequest.status is updated to SENTwith OneKey or Veeva request and response details or DS_ACTION_REQURIED with all Reltio details
      • Otherwise FAILEDstatus is recorded in DCRRequestwith a detailed error message.
        • Mongo Update:  DCRRequest.status is updated to FAILED with all required attributes, request, and exception response details
   5. Initialize Workflow in Reltio (only requests that TargetTypeis Reltio)
      1. POST /workflow/_initiate operation is invoked to init new Workflow in Reltio

         Workflow attributes	Mapping
         changeRequest.uri	ChangeRequest Reltio URI
         changeRequest.changes	Entity URI

   6. Then Auto close logic is invoked to evaluate whether DCR request meets conditions to be auto accepted or auto rejected. Logic is based on decision table PreCloseConfig. If DCRRequest.country is contained in PreCloseConfig.acceptCountries or PreCloseConfig.rejectCountries then DCR is accepted or rejected respectively.
   7. return DCRResponse to Client - During the flow, DCRRespone may be returned to Client with the specific errorCode or requestStatus. The description for all response codes is presented on this page: Get DCR status

Triggers

Trigger action	Component	Action	Default time
REST call	DCR Service: POST /dcr	create DCRs in the Reltio, OneKey or Veeva system	API synchronous requests - realtime

Dependent components

Component	Usage
DCR Service	Main component with flow implementation
OK DCR Service	OneKey Adapter - API operations
Veeva DCR Service	Veeva Adapter - API operations and S3/SFTP communication
Manager	Reltio Adapter - API operations
Hub Store	DCR and Entities Cache

---

DCR state change

Page ID: 218438617 Page Link: /display/GMDM/DCR+state+change

Description

The following diagram represents the DCR state changes. DCR object stat is saved in HUB and in Reltio DCR entity object. The state of the DCR is changed based on the Reltio/IQVIA/Veeva Data Steward action.

Flow diagram

Steps

1. DCR is created (OPEN)  - Create DCR
   1. DCR is sent to Reltio, OneKey or Veeva
      1. When sent to Reltio
         1. Pre Close logic is invoked to auto accept (PRE_ACCEPT) or auto reject (PRE_REJECT) DCR
         2. Reltio Data Steward process the DCR - Reltio: process DCR Change Events
      2. OneKey Data Steward process the DCR - OneKey: process DCR Change Events
      3. Veeva Data Steward process the DCR - Veeva: process DCR Change Events

Data Steward DCR status change perspective

Transaction Log

There are the following main assumptions regarding the transaction log in DCR service:

• Main transaction
  • The user sends to the DCR service list of the DCR Requests and receives the list of the DCR Responses
    • Transaction service generates the transaction ID for the input request - this is used as the correlation ID for each separated DCR Request in the list
    • Transaction service save:
      • METADATA
        • main transaction ID
        • userName
        • extDCRRequestIds (list of all)
      • BODY
        • the DCR Requests list and the DCR Response List
• State change transaction
  • DCR object state may change depending on the DS decision, for each state change (represented as a green box in the above diagram) the transaction is saved with the following attributes:
    • Transaction METADATA
      • main transaction ID
      • extDCRRequestId
      • dcrRequestId
      • Reltio:
        • VRStatus
        • VRStatusDetail
      • HUB:
        • DCRRequestStatusDetails
      • optionally:
        • errorMessage

        • errorCode
          

    • Transaction BODY:
      • Input Event

Log appenders:

• Kafka Transaction appender - saves whole events(metadata+body) to Kafka - data presented in the Kibana Dashboard
• Simple Transaction logger - saves the transactions details to the file in the following format:
  • {ID}    {extDCRRequestId}   {dcrRequestId}   {VRStatus}   {VRStatusDetail}   {DCRRequestStatusDetails}   {errorCode}   {errorMessage}

Triggers

Trigger action	Component	Action	Default time
REST call	DCR Service: POST /dcr	create DCRs in the Reltio system or in OneKey	API synchronous requests - realtime
IN Events incoming	dcr-service-2:DCRReltioResponseStream	process publisher full change request events in the stream	realtime: events stream processing
IN Events incoming	dcr-service-2:DCROneKeyResponseStream	process publisher full change request events in the stream	realtime: events stream processing
IN Events incoming	dcr-service-2:DCRVeevaResponseStream	process publisher full change request events in the stream	realtime: events stream processing

Dependent components

Component	Usage
DCR Service	Main component with flow implementation
OK DCR Service	OneKey Adapter  - API operations
Veeva DCR Service	Veeva Adapter  - API operations
Manager	Reltio Adapter  - API operations
Hub Store	DCR and Entities Cache

---

Get DCR status

Page ID: 209949187 Page Link: /display/GMDM/Get+DCR+status

Description

The client creates DCRs in Reltio, OneKey or Veeva OpenData using the Create DCR operation. The status is then asynchronously updated in the DCR Registry. The operation retrieves the current status of the DCRs that the updated date is between 'updateFrom' and 'updateTo' input parameters. PforceRx first asks what DCRs have been changed since last time they checked (usually 24h) and then iterate for each DCR they get detailed info.

Flow diagram

,

Source: Lucid

Dependent flows:

1. The DCRRegistry is enriched by the DCR events that are generated by Reltio - the flow description is here - Reltio: process DCR Change Events
2. The DCRRegistry is enriched by the DCR events generated in OneKey DCR service component - after submitVR operation is invoked to ONEKEY, each DCR is traced asynchronously in this process - OneKey: process DCR Change Events
3. The DCRRegistry is enriched by the DCR events generated in Veeva OpenData DCR service component - after submitVR operation is invoked to VEEVA, each DCR is traced asynchronously in this process - Veeva: process DCR Change Events

Steps

Status

There are the following request statuses that users may receive during Create DCR operation or during checking the updated status using GET /dcr/_status operation described below:

RequestStatus	DCRStatus	Internal Cache status	Description
REQUEST_ACCEPTED	CREATED	SENT_TO_OK	DCR was sent to the ONEKEY system for validation and pending the processing by Data Steward in the system
REQUEST_ACCEPTED	CREATED	SENT_TO_VEEVA	DCR was sent to the VEEVA system for validation and pending the processing by Data Steward in the system
REQUEST_ACCEPTED	CREATED	DS_ACTION_REQUIRED	DCR is pending Data Steward validation in Reltio, waiting for approval or rejection
REQUEST_ACCEPTED	CREATED	OK_NOT_FOUND	Used when ONEKEY profile was not found after X retries
REQUEST_ACCEPTED	CREATED	VEEVA_NOT_FOUND	Used when VEEVA profile was not found after X retries
REQUEST_ACCEPTED	CREATED	WAITING_FOR_ETL_DATA_LOAD	Used when waiting for actual data profile load from 3rd Party to appear in Reltio
REQUEST_ACCEPTED	ACCEPTED	ACCEPTED	Data Steward accepted the DCR, changes were applied
REQUEST_ACCEPTED	ACCEPTED	PRE_ACCEPTED	PreClose logic was invoked and automatically accepted DCR according to decision table in PreCloseConfig
REQUEST_REJECTED	REJECTED	REJECTED	Data Steward rejected the changes presented in the Change Request
REQUEST_REJECTED	REJECTED	PRE_REJECTED	PreClose logic was invoked and automatically rejected DCR according to decision table in PreCloseConfig
REQUEST_FAILED	-	FAILED	DCR requests failed due to: validation error/ unexpected error e.t.d - details in the errorCode and errorMessage

Error codes:

There are the following classes of exception that users may receive during Create DCR operation:

Class	errorCode	Description	HTTP code
1	DUPLICATE_REQUEST	request rejected - extDCRRequestId  is registered - this is a duplicate request	403
2	NO_CHANGES_DETECTED	entities are the same (request is the same) - no changes	400
3	VALIDATION_ERROR	ref object does not exist (not able to find HCP/HCO target object	404
3	VALIDATION_ERROR	ref attribute does not exist - not able to find nested attribute in the target object	400
3	VALIDATION_ERROR	wrong number of HCP/HCO entities in the input request	400

1. Clients execute the API GET/dcr/_status request
2. Kong receives requests and handles authentication
3. If the authentication succeeds the request is forwarded to the dcr-service-2 component,
4. DCR Service checks permissions to call this operation and the correctness of the request, then the flow is started and the following steps are executed
   1. Query on mongo is executed to get all DCRs matching input parameters:
      1. updateFrom (date-time) - DCR last update from - DCRRequestDetails.status.changeDate
      2. updateTo (date-time) - DCR last update to - DCRRequestDetails.status.changeDate
      3. limit (int) the maximum number of results returned through API - the recommended value is 25. The max value for a single request is 50.
      4. offset(int) - result offset - the parameter used to query through results that exceeded the limit.
   2. Resulted values are aggregated and returned to the Client.
   3. The client receives the List body.

Triggers

Trigger action	Component	Action	Default time
REST call	DCR Service: GET/dcr/_status	get status of created DCRs. Limit the results using query parameters like dates and offset	API synchronous requests - realtime

Dependent components

Component	Usage
DCR Service	Main component with flow implementation
Hub Store	DCR and Entities Cache

---

OneKey: create DCR method (submitVR) - direct

Page ID: 209949294 Page Link: /display/GMDM/OneKey%3A+create+DCR+method+%28submitVR%29+-+direct

Description

Rest API method exposed in the OK DCR Service component responsible for submitting the VR to OneKey

Flow diagram

Steps

1. Receive the API request
2. Validate - check if the onekey crosswalk exists once there is an update on the profile, otherwise reject the request
3. The DCR is mapped to OK VR Request and it's submitted using API REST method POST /vr/submit. (mapping described below)
   1. If the submission is successful then:
      • DCRRequesti updated to SENT_TO_OKwith OK request and response details. DCRRegistryONEKEY collection in saved for tracing purposes. The process that reads and check ONEKEY VRs is described here: OneKey: generate DCR Change Events (traceVR)
   2. Otherwise FAILEDstatus is recorded and the response is returned with an OK error response

Mapping

VR - Business Fields Requirements_UK.xlsx - file that contains VR UK requirements and mapping to IQVIA model

HUB					ONEKEY
	attributes	attributes	codes		mandatory	attributes	values
HCO
					Y	entityType	WORKPLACE
					Y	validation.clientRequestId	HUB_GENERATED_ID
					Y	validation.process	Q
					Y	validation.requestDate	1970-01-01T00:00Z
					Y	validation.callDate	1970-01-01T00:00Z
attributes					Y	validation.requestProcess	I
	extDCRComment					validation.requestComment
	country				Y	isoCod2
	reference Entity	crosswalk	ONEKEY			workplace.workplaceEid
	name					workplace.usualName
						workplace.officialName
	otherHCOAffiliations	parentUsualName				workplace.parentUsualName
	subTypeCode		COTFacilityType (TET.W.*)			workplace.typeCode
	typeCode	no value in PFORCERX	HCOSubType (LEX.W.*)			workplace.activityLocationCode
	addresses
		sourceAddressId				N/A
		addressType				N/A
		addressLine1				address.longLabel
		addressLine2				address.longLabel2
		addressLine3				N/A
		stateProvince	AddressState (DPT.W.*)			address.countyCode
		city			Y	address.city
		zip				address.longPostalCode
		country			Y	address.country
		rank				get address with rank=1
	emails
		type				N/A
		email				workplace.email
		rank				get email with rank=1
	otherHCOAffiliations
		type				N/A
		rank				get affiliation with rank=1
	reference Entity	otherHCOAffiliations reference entity onekeyID	ONEKEY			workplace.parentWorkplaceEid
	phones
		type	contains FAX
		number				workplace.telephone
		rank				get phone with rank=1
		type	not contains FAX
		number				workplace.fax
		rank				get phone with rank=1
HCP
					Y	entityType	ACTIVITY
					Y	validation.clientRequestId	HUB_GENERATED_ID
					Y	validation.process	Q
					Y	validation.requestDate	1970-01-01T00:00Z
					Y	validation.callDate	1970-01-01T00:00Z
attributes					Y	validation.requestProcess	I
	extDCRComment					validation.requestComment
	country				Y	isoCod2
	reference Entity	crosswalk	ONEKEY			individual.individualEid
	firstName					individual.firstName
	lastName				Y	individual.lastName
	middleName					individual.middleName
	typeCode					N/A
	subTypeCode		HCPSubTypeCode (TYP..*)			individual.typeCode
	title		HCPTitle (TIT.*)			individual.titleCode
	prefix		HCPPrefix (APP.*)			individual.prefixNameCode
	suffix					N/A
	gender		Gender (.*)			individual.genderCode
	specialties
		typeCode	HCPSpecialty (SP.W.*)			individual.speciality1
		type				N/A
		rank				get speciality with rank=1
		typeCode	HCPSpecialty (SP.W.*)			individual.speciality2
		type				N/A
		rank				get speciality with rank=2
		typeCode	HCPSpecialty (SP.W.*)			individual.speciality3
		type				N/A
		rank				get speciality with rank=3
	addresses
		sourceAddressId				N/A
		addressType				N/A
		addressLine1				address.longLabel
		addressLine2				address.longLabel2
		addressLine3				N/A
		stateProvince	AddressState (DPT.W.*)			address.countyCode
		city			Y	address.city
		zip				address.longPostalCode
		country			Y	address.country
		rank				get address with rank=1
	identifiers
		type				N/A
		id				N/A
	phones
		type				N/A
		number				individual.mobilePhone
		rank				get phone with rank=1
	emails
		type				N/A
		email				individual.email
		rank				get phone with rank=1
---	---	---	---	---	---	---	---
	contactAffiliations	no value in PFORCERX
		type	RoleType (TIH.W.*)			activity.role
		primary				N/A
		rank				get affiliation with rank=1
	contactAffiliations reference Entity	crosswalks	ONEKEY			workplace.workplaceEid
HCP & HCO
					Y	entityType	ACTIVITY
	For HCP full mapping check the HCP section above				Y	validation.clientRequestId	HUB_GENERATED_ID
	For HCO full mapping check the HCO section above				Y	validation.process	Q
					Y	validation.requestDate	1970-01-01T00:00Z
					Y	validation.callDate	1970-01-01T00:00Z
attributes					Y	validation.requestProcess	I
	extDCRComment					validation.requestComment
	country				Y	isoCod2
	addresses
		If the HCO address exists map to ONEKEY address				address (mapping HCO)
		else
		If the HCP address exists map to ONEKEY address				address (mapping HCP)
	contactAffiliations	no value in PFORCERX
		type	RoleType (TIH.W.*)			activity.role
		primary				N/A
		rank				get affiliation with rank=1

Triggers

Trigger action	Component	Action	Default time
REST call	DCR Service: POST /dcr	create DCRs in the ONEKEY	API synchronous requests - realtime

Dependent components

Component	Usage
DCR Service 2	Main component with flow implementation
Hub Store	DCR and Entities Cache

---

OneKey: generate DCR Change Events (traceVR)

Page ID: 209950500 Page Link: /pages/viewpage.action?pageId=209950500

Description

This process is triggered after the DCR was routed to Onekey based on the decision table configuration. The process of tracing the VR changes is based on the OneKey VR changes. During this process HUB, DCR Cache is triggered every hour for SENT DCR's and check VR status using OneKey web service. After verification, the DCR Change event is generated. The DCR event is processed in the OneKey: process DCR Change Events and the DCR is updated in Reltio with Accepted or Rejected status.

Flow diagram

Steps

• Every N  hours OK VR requests with status SENTare queried in DCRRegistryONEKEY store*.*
• For each open requests, its status is checked it OK using REST API method /vr/trace
• The first check is the VR.rsp.statusattribute, checking if the status is SUCCESS
• Next, if the process status (VR.rsp.results.processStatus) is REQUEST_PENDING_OKE | REQUEST_PENDING_JMS | REQUEST_PROCESSED or OK data export date (VR.rsp.results.trace6CegedimOkcExportDate) is earlier than 24 hours then the processing of the request is postponed to the next check
  • exportDate or processStatusare optional and can be null.
  • The process goes to the next step only if processStatusis  REQUEST_RESPONDED | RESPONSE_SENT
  • The process is blocked to next check only if  trace6CegedimOkcExportDateis not null and is earlier than 24h
• If the processStatusis validated and VR.rsp.results*.responseStatus*is VAS_NOT_FOUND | VAS_INCOHERENT_REQUEST | VAS_DUPLICATE_PROCESS then OneKeyDCREvent is being generated with status REJECTED

  OneKeyChangeRequest attributes	Mapping
  vrStatus	"CLOSED"
  vrStatusDetail	"REJECTED"
  traceResponseReceivedDate	current time
  oneKeyComment	OK.responseComment

• Next.
  • if responseStatusisVAS_FOUND | VAS_FOUND_BUT_INVALID then OneKeyDCREvent is being generated with status ACCEPTED. ( now the new ONEKEY profile will be loaded to Reltio using ETL data load. The OneKey: process DCR Change Events is processing this events ad checks the Reltio if the ONEKEY is created and PfizerCustomerGlobalId is assigned, this process will wait until ONEKEY is in Reltio so the client received the ACCEPTED DCR only after this condition is met)

    DCR entity attributes	Mapping
    vrStatus	"CLOSED"
    vrStatusDetail	"ACCEPTED"
    traceResponseReceivedDate	current time
    oneKeyComment	OK.responseComment \n ONEKEY ID = individualEidValidated or workplaceEidValidated

  • events are published to the $env-internal-onekey-dcr-change-events-in topic

Triggers

Trigger action	Component	Action	Default time
IN Timer (cron)	dcr-service:TraceVRService	query mongo to get all SENT DCR's related to the PFORCERX process	every hour
OUT Events	dcr-service:TraceVRService	generate the OneKeyDCREvent	every hour

Dependent components

Component	Usage
DCR Service	Main component with flow implementation
Hub Store	DCR and Entities Cache

---

OneKey: process DCR Change Events

Page ID: 209949303 Page Link: /display/GMDM/OneKey%3A+process+DCR+Change+Events

Description

The process updates the DCR's based on the Change Request events received from [ONEKEY|VOD] (after trace VR method result). Based on the [IQVIA|VEEVA] Data Steward decision the stateattribute contains relevant information to update DCR status. During this process also the comments created by IQVIA DS are retrieved and the relationship (optional step) between the DCR object and the newly created entity is created. DCR status is accepted only after the [ONEKEY|VOD] profile is created in Reltio, only then the Client will receive the ACCEPTED status. The process is checking Reltio with delay and retries if the ETL load is still in progress waiting for [ONEKEY|VOD] profile.

Flow diagram

OneKey variant

Veeva variant:

Steps

• OneKey: generate DCR Change Events (traceVR) publishes simple events to $env-internal-onekey-dcr-change-events-in: DCR_CHANGED
  • Veeva specific: Veeva: generate DCR Change Events (traceVR) publishes simple events to $env-internal-veeva-dcr-change-events-in: DCR_CHANGED
• Events are aggregated in a time window (recommended the window length 24 hours) and the last event is returned to the process after the window is closed.
• Events are processed in the Stream and based on the OneKeyDCREvent.OneKeyChangeRequest``.vrStatus | VeevaDCREvent.VeevaChangeRequestDetails.vrStatus attribute decision is made
• DCR is retrieved from the cache based on the _id of the DCR
• If the event state is ACCEPTED
  • Get Reltio entity PfizerCustomerID by [ONEKEY|VOD] crosswalk
  • If such crosswalk entity exists in Reltio:
    • PfizerGlobalCustomerId is saved in Registry and will be returned to the Client
    • During the process, the optional check is triggered - create the relation between the DCR object and newly created entities
      • if DCRRegistry contain an empty list of entityUris, or some of the newly created entity is not present in the list, the Relation between this object and the DCR has to be created
        • DCRentity is updated in Reltio and the relation between the processed entity and the DCR entity
          • Reltio source name (crosswalk. type): DCR
          • Reltio relation type: HCPtoDCRor HCOtoDCR(depending on the object type)
        • Newly created entities uris should be retrieved by the individualEidValidated or workplaceEidValidated (it may be both) attributes from the events that represent the HCP or HCO crosswalks.
    • The status in Reltio and in Mongo is updated

      DCR entity attributes	Mapping for OneKey	Mapping for Veeva
      VRStatus	CLOSED
      VRStatusDetail	state: ACCEPTED
      Comments	ONEKEY comments ({VR.rsp.responseComments}) ONEKEY ID = i``ndividualEidValidated or workplaceEidValidated	VEEVA comments = VR.rsp.responseComments VEEVA ID = entityUris
      PfizerGlobalCustomerId	This is required in ACCEPTED status

  • If the [ONEKEY|VOD] does not exist in Reltio
    • Regenerate the Event with a new timestamp to the input topic so this will be processed in the next hours
    • Update the Reltio DCR status

      • DCR entity attributes	Mapping
        VRStatus	OPEN
        VRStatusDetail	ACCEPTED

    • update the Mongo status to the OK_NOT_FOUND | VEEVA_NOT_FOUND and increase the "retryCounter" attribute
• If the event state is REJECTED

  • If a Reltio DS has already seen this request, REJECT the DCR and end the flow (if the initial target type is Reltio)

    The status in Reltio and in Mongo is updated

    DCR entity attributes	Mapping
    VRStatus	CLOSED
    VRStatusDetail	state: REJECTED
    Comments	`[ONEKEY

  • If this is based on the routing table and it was never sent to the Reltio DS, then create the DCR workflow and send this to the Reltio DS. Add the information comment that this was Rejected by the OneKey, so now Reltio DS has to decide if this should be REJECTED or APPLIED in Reltio. Add the comment that this is not possible to execute the sendTo3PartyValidation button in this case. Steps:

    • Check if the initial target type is [ONEKEY|VOD]
    • Use the DCR Request that was initially received from PforceRx and is a Domain Model request (after validation)
    • Send the DCR to Reltio the service returns the following response:
      • ACCEPTED (change request accepted by Reltio)
        • update the status to DS_ACTION_REQUIEREDand in the comment add the following: "This DCR was REJECTED by the [ONEKEY|VOD] Data Steward with the following comment: <[ONEKEY|VOD] reject comment>. Please review this DCR in Reltio and APPLY or REJECT. It is not possible to execute the sendTo3PartyValidation button in this case"
        • initialize new Workflow in Reltio with the comment.
        • save data in the DCR entity status in Reltio and update Mongo DCR Registry with workflow ID and other attributes that were used in this Flow.
      • REJECTED  (failure or error response from Reltio)
        • CLOSE the DCR with the information that DCR was REJECTED by the [ONEKEY|VOD] and Reltio also REJECTED the DCR. Add the error message from both systems in the comment.

Triggers

Trigger action	Component	Action	Default time
IN Events incoming	dcr-service-2:DCROneKeyResponseStream dcr-service-2:DCRVeevaResponseStream ($env-internal-veeva-dcr-change-events-in)	process publisher full change request events in the stream	realtime: events stream processing

Dependent components

Component	Usage
DCR Service 2	Main component with flow implementation
Manager	Reltio Adapter  - API operations
Publisher	Events publisher generates incoming events
Hub Store	DCR and Entities Cache

---

Reltio: create DCR method - direct

Page ID: 209949292 Page Link: /display/GMDM/Reltio%3A+create+DCR+method+-+direct

Description

Rest API method exposed in the Manager component responsible for submitting the Change Request to Reltio

Flow diagram

Steps

1. Receive the DCR request generated by DCR Service 2 component
2. Depending on the Action execute the method in the Manager component:
   1. insert - Execute standard Create/Update HCP/HCO/MCO operation with additional changeRequest.id parameter
   2. update - Execute Update Attributes operation with additional changeRequest.id parameter
      1. the combination of IGNORE_ATTRIBUTE & INSERT_ATTRIBUTE once updating existing parameter in Reltio
      2. the INSERT_ATTRIBUTE once adding new attribute to Reltio
   3. delete - Execute Update Attribute operation with additional changeRequest.id parameter
      1. the UPDATE_END_DATE on the entity to inactivate this profile
3. Based on the Reltio response the DCR Response is returned:
   1. REQUEST_ACCEPTED - Reltio processed the request successfully
   2. REQUEST_FAILED - Reltio returned the exception, Client will receive the detailed description in the errorMessage

Triggers

Trigger action	Component	Action	Default time
REST call	DCR Service: POST /dcr2	Create change Requests in Reltio	API synchronous requests - realtime

Dependent components

Component	Usage
DCR Service	Main component with flow implementation
Hub Store	DCR and Entities Cache

---

Reltio: process DCR Change Events

Page ID: 209949300 Page Link: /display/GMDM/Reltio%3A+process+DCR+Change+Events

Description

The process updates the DCR's based on the Change Request events received from Reltio(publishing). Based on the Data Steward decision the stateattribute contains relevant information to update DCR status. During this process also the comments created by DS are retrieved and the relationship (optional step) between the DCR object and the newly created entity is created.

Flow diagram

Steps

• Event publisher publishes simple events to $env-internal-reltio-dcr-change-events-in: DCR_CHANGED("CHANGE_REQUEST_CHANGED") and DCR_REMOVED("CHANGE_REQUEST_REMOVED")

• When the events do not contain the ThirdPartyValidation flag it means that DS APPLIED or REJECTED the DCR, the following logic is applied

  • Events are processed in the Stream and based on the targetChangeRequest.state attribute decision is made
    • If the state is APPLIED or REJECTS, DCR is retrieved from the cache based on the changeRequestURI
      • If DCR exists in Cache The status in Reltio is updated

        DCR entity attributes	Mapping
        VRStatus	CLOSED
        VRStatusDetail	state: APPLIED → ACCEPTED state: REJECTED → REJECTED

      • Otherwise, the events are rejected and the transaction is ended
    • The PfizerCustomerGlobalId is retrieved for newly created entities in Reltio based on the main entity URI.
    • During the process, the optional check is triggered - create the relation between the DCR object and newly created entities
      • if DCRRegistry contain an empty list of entityUris, or some of the newly created entity is not present in the list, the Relation between this object and the DCR has to be created
        • DCRentity is updated in Reltio and the relation between the processed entity and the DCR entity
          • Reltio source name (crosswalk. type): DCR
          • Reltio relation type: HCPtoDCRor HCOtoDCR(depending on the object type)
    • The comments added by the DataSteward during the processing of the Change request is retrieved using the following operation:
      • GET /tasks?objectURI=entities/
      • The processInstanceComments is retrieved from the response and added to DCRRegistry.changeRequestComment

• Otherwise, when the events contain the ThirdPartyValidation flag it means that DS decided to send the DCR to IQVIA or VEEVA for the validation, the following logic is applied:

  • If the current targetType is ONEKEY | VEEVA
    • REJECT the DCR and add the comment on the DCR in Retlio that "DCR was already processed by [ONEKEY|VEEVA] Data Stewards, REJECT because it is not allowed to send this DCR one more time to [IQVIA|VEEVA]"
  • If the current targetType is Reltio, it means that we can send this DCR to [IQVIA|VEEVA] for validation 
    • Use the DCR Request that was initially received from PforceRx and is a Domain Model request (after validation)
    • Execute the POST /dcr method in [ONEKEY|VEEVA] DCR Service, the service returns the following response:
      • ACCEPTED - update the status to [SENT_TO_OK|SENT_TO_VEEVA]
      • REJECTED - it means that some unexpected exception occurred in [ONEKEY|VEEVA], or request was rejected by [ONEKEY|VEEVA], or the ONEKEY crosswalk does not exist in Reltio, and [ONEKEY|VEEVA]service rejected this request
        • Veeva specific: When VOD crosswalk does not exist in Reltio, current version of profile is being sent to Veeva for validation independent from initial changes which where incorporated within DCR

Triggers

Trigger action	Component	Action	Default time
IN Events incoming	dcr-service-2:DCRReltioResponseStream	process publisher full change request events in the stream	realtime: events stream processing

Dependent components

Component	Usage
DCR Service DCR Service 2	Main component with flow implementation
Manager	Reltio Adapter  - API operations
Publisher	Events publisher generates incoming events
Hub Store	DCR and Entities Cache

---

Reltio: Profiles created by DCR

Page ID: 510266969 Page Link: /display/GMDM/Reltio%3A+Profiles+created+by+DCR

DCR type	Approval/Reject	Record visibility in MDM	Crosswalk Type	Crosswalk Value	Source
DCR create for HCP/HCO	Approved by OneKey/VOD	HCP/HCO created in MDM	ONEKEY	VOD	onekey id
Approved by DSR	HCP/HCO created in MDM	System source name from DCR (KOL_OneView, PforceRx, etc)	DCR ID	System source name from DCR (KOL_OneView, PforceRx, etc)
DCR edit for HCP/HCO	Approved by OneKey/VOD	HCP/HCO requested attribute updated in MDM	ONEKEY	VOD
Approved by DSR	HCP/HCO requested attribute updated in MDM	Reltio	entity uri	Reltio
DCR edit for HCPaddress/HCO address	Approved by OneKey/VOD	New address created in MDM, existing address marked as inactive	ONEKEY	VOD
Approved by DSR	New address created in MDM, existing address marked as inactive	Reltio	entity uri	Reltio

---

Veeva DCR flows

Page ID: 379332475 Page Link: /display/GMDM/Veeva+DCR+flows

Description

The process is responsible for creating DCRs which are stored (Store VR) to be further transferred and processed by Veeva. Changes can be suggested by the DS using "Suggest" operation in Reltio and "Send to Third Party Validation" button. All DCRs are saved in the dedicated collection in HUB Mongo DB, required to gather metadata and trace the changes for each DCR request. During this process, the communication to Veeva Opendata is established via S3/SFTP communication. SubmitVR operation is executed to create a new ZIP files with DCR requests spread across multiple CSV files. The TraceVR operation is executed to check if Veeva responded to initial DCR Requests via ZIP file placed Inbound S3 dir.

The process is divided into 3 sections:

1. Create DCR request - Veeva
2. Submit DCR Request - Veeva
3. Trace Validation Request - Veeva

The below diagram presents an overview of the entire process. Detailed descriptions are available in the separated subpages.

Business process diagram for R1 phase

Flow diagram

Steps

• CreateVR
  • Process of saving DCR requests in Mongo Cache after being triggered by DCR Service 2.
  • DCR request information is translated to Veeva's model and stored in dedicated collection for Veeva DCRs.
• SubmitVR
  • The process of submitting VR stored in Mongo Cache to Veeva's SFTP via S3 bucket. The process aggregates events stored in Mongo Cache since last submit.
  • New ZIP is created with CSV files containing DCR request for Veeva. ZIP is placed in outbound dir in S3 bucket which is further synchronized to Veeva's SFTP.
  • Each DCR is updated with ZIP file name which was used to transfer request to Veeva.
• TraceVR
  • The process of tracing VR is triggered each hours by Spring Scheduler.
  • Inbound S3 bucket is searched for ZIP files with CSVs containing DCR responses from Veeva. There are multiple dirs in S3 buckets, each for specific group of countries (currently CN and APAC).
  • Parts of DCR responses are spread across multiple files. Combined information is being processed.
  • Finally information about DCR is updated in Mongo Cache and events are produced to dedicated topic for DCR Service 2 for further processing.

Triggers

DCR service 2 is being triggered via /dcr API calls which are triggered by Data Stewards actions (R1 phase) → "Suggests 3rd party validation" which pushes DCR from Reltio to HUB.

Dependent components

Described in the separated sub-pages for each process.

Design document for HUB development

1. Design → VeevaOpenData-implementation.docx
2. Reltio HUB-VOD mapping → VeevaOpenDataAPACDataDictionary.xlsx
3. VOD model description (v4) → Veeva_OpenData_APAC_Data_Dictionary v4.xlsx

---

Create DCR request - Veeva

Page ID: 386814533 Page Link: /display/GMDM/Create+DCR+request+-+Veeva

Description

The process of creating new DCR requests to the Veeva OpenData. During this process, new DCRs are created in DCRregistryVeeva mongo collection.

Flow diagram

Steps

• Service is called by Rest API
• Input request is validated. If request is invalid - return response with status REJECTED
• Transform input request to Veeva DCR model
  • translate lookup codes to Veeva source codes
  • fill the Veeva DCR model with input request values
• Save DCR request to DCRRegistryVeeva mongo collection with status NEW

Mappings

DCR domain model→ VOD mapping file: VeevaOpenDataAPACDataDictionary-mmor-mapping.xlsx

Veeva integration guide

Submit DCR Request - Veeva

Page ID: 379333348 Page Link: /display/GMDM/Submit+DCR+Request+-+Veeva

Description

The process of submitting new validation requests to the Veeva OpenData service via VeevaAdapter (communication with S3/SFTP) based on DCRRegistryVeeva mongo collection . During this process, new DCRs are created in VOD system.

Flow diagram

Steps

Veeva DCR service flow:

• Every N hours Veeva DCR requests with status NEW are queried in DCRRegistryVeeva store.
• DCR are group by country
• For each country:
  • • merge Veeva DCR requests - create one zip file for each country
    • upload zip file to S3 location
    • update DCR status to SENT if upload status is successful

      DCR entity attributes	Mapping
      DCRID	Veeva VR Request Id
      VRStatus	"OPEN"
      VRStatusDetail	"SENT"
      CreatedBy	MDM HUB
      SentDate	current time

SFTP integration service flow:

• Every N  hours grab all zip files from S3 locations
• Upload files to corresponding SFTP server

Triggers

Trigger action	Component	Action	Default time
Spring scheduler	mdm-veeva-dcr-service:VeevaDCRRequestSender	prepare ZIP files for VOD system	Called every specified interval

Dependent components

Component	Usage
Veeva adapter	Upload DCR request to s3 location

---

Trace Validation Request - Veeva

Page ID: 379333358 Page Link: /display/GMDM/Trace+Validation+Request+-+Veeva

Description

The process of tracing the VR changes based on the Veeva VR changes. During this process HUB, DCRRegistryVeeva Cache is triggered every hour for SENT DCR's and check VR status using Veeva Adapter (s3/SFTP integration). After verification DCR event is sent to DCR Service 2  Veeva response stream.

Flow diagram

Steps

• Every N get all Veeva DCR responses using Veeva Adapter
• For each response:

  • check if status is terminal - (CHANGE_ACCEPTED, CHANGE_PARTIAL, CHANGE_REJECTED, CHANGE_CANCELLED)

    • if not - go to next response

  • query DCRregistryVeeva mongo collection for DCR with given key and SENT status

  • get Veeva ID (vid__v) from response file

  • generate Veeva DCR change event

  • update DCR status in DCRRegistryVeeva mongo collection

    • resolution is CHANGE_ACCEPTED, CHANGE_PARTIAL

      DCR entity attributes	Mapping
      VRStatus	"CLOSED"
      VRStatusDetail	"ACCEPTED"
      ResponseTime	veeva response completed date
      Comments	veeva response resolution notes

    • resolution is CHANGE_REJECTED, CHANGE_CANCELLED

      DCR entity attributes	Mapping
      VRStatus	"CLOSED"
      VRStatusDetail	"REJECTED"
      ResponseTime	veeva response completed date
      Comments	veeva response resolution notes

Triggers

Trigger action	Component	Action	Default time
IN Spring scheduler	mdm-veeva-dcr-service:VeevaDCRRequestTrace	start trace validation request process	every hour
OUT Kafka topic	mdm-dcr-service-2:VeevaResponseStream	update DCR status in Reltio, create relations	invokes Kafka producer for each veeva DCR response

Dependent components

Component	Usage
DCR Service 2	Process response event

---

Veeva: create DCR method (storeVR)

Page ID: 379332642 Page Link: /pages/viewpage.action?pageId=379332642

Description

Rest API method exposed in the Veeva DCR Service component responsible for creating new DCR requests specific to Veeva OpenData (VOD) and storing them in dedicated collection for further submit. Since VOD enables communication only via S3/SFTP, it's required to use dedicated mechanism to actually trigger CSV/ZIP file creation and file placement in outbound directory. This will periodic call to Submit VR method will be scheduled once a day (with cron) which will in the end call VeevaAdapter with method createChangeRequest.

Flow diagram

Steps

1. Receive the API request
2. Validate initial request
   1. check if the Veeva crosswalk exists once there is an update on the profile
   2. otherwise it's required to prepare DCR to create new Veeva profile
   3. If there is any formal attribute missing or incorrect: skip request
3. Then the DCR is mapped to Veeva Request by invoking mapper between HUB DCR → VEEVA model
   1. For mapping purpose below mapping table should be used
   2. If there is not proper LOV mapping between HUB and Veeva, default fallback should be set to question mark → ?
4. Once proper request has been created,it should be stored as a VeevaVRDetails entry in dedicated DCRRegistryVeeva collection to be ready for actually send via Submit VR job and for future tracing purposes
5. Prepare return response for initial API request with below logic
   1. Generate sample request after successful mongo insert →  generateResponse(dcrRequest, RequestStatus.REQUEST_ACCEPTED, null, null)
   2. Generate error when validation or exception →  generateResponse(dcrRequest, RequestStatus.REQUEST_FAILED, getErrorDetails(), null);

Mapping HUB DCR → Veeva model

• Below table does not contain all new attributes which are new in Reltio. Only the most important ones were mentioned there.
• File STTM Stats_SG_HK_v3.xlsx contains full mapping requirements from Veeva OpenData to Reltio data model. It does contain full data mapping which should be covered in target DCR process for VOD.

Reltio		HUB		VEEVA
Attribute Path	Details	DCR Request path	Details	File Name	Field Name	Required for Add Request?	Required for Change Request?	Description	Reference (RDM/LOV)	NOTE
HCO
N/A		Mongo Generated ID for this DCR	Kafka KEY	once mapping from HUB Domain DCRRequest take this from DCRRequestD.dcrRequestId: String, // HUB DCR request id - Mongo ID - required in ONEKEY service	change_request	dcr_key	Y	Y	Customer's internal identifier for this request
Change Requests comments		extDCRComment		change_request	description	Y	Y	Requester free-text comments explaining the DCR
targetChangeRequest.createdBy		createdBy		change_request	created_by	Y	Y	For requestor identification
N/A		if new objects - ADD, if veeva ID CHANGE		change_request	change_request_type	Y	Y	ADD_REQUEST or CHANGE_REQUEST
N/A	depends on suggested changes (check use-cases)	main entity object type HCP or HCO		change_request	entity_type	Y	N	HCP or HCO	EntityType
N/A		Mongo Generated ID for this DCR	Kafka KEY		change_request_hco	dcr_key	Y	Y	Customer's internal identifier for this request
Reltio Uri and Reltio Type	when insert new profile	entities.HCO.updateCrosswalk.type (Reltio) entities.HCO.updateCrosswalk.value (Reltio id) and refId.entityURI	concatenate Reltio:rvu44dm	change_request_hco	entity_key	Y	Y	Customer's internal HCO identifier
Crosswalks - VEEVA crosswalk	when update on VEEVA	entities.HCO.updateCrosswalk.type (VEEVA) entities.HCO.updateCrosswalk.value (VEEVA ID)		change_request_hco	vid__v	Y	N	Veeva ID of existing HCO to update; if blank, the request will be interpreted as an add request
configuration/entityTypes/HCO/attributes/OtherNames/attributes/Name	first element	TODO - add new attribute		change_request_hco	alternate_name_1__v	Y	N
??		??		change_request_hco	business_type__v	Y	N		HCOBusinessType	TO BE CONFIRMED
configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/FacilityType		HCO.subTypeCode		change_request_hcp	major_class_of_trade__v	N	N		COTFacilityType	In PforceRx - Account Type, more info: MR-9512 - Getting issue details... STATUS
configuration/entityTypes/HCO/attributes/Name		name		change_request_hco	corporate_name__v	N	Y
configuration/entityTypes/HCO/attributes/TotalLicenseBeds		TODO - add new attribute		change_request_hco	count_beds__v	N	Y
configuration/entityTypes/HCO/attributes/Email/attributes/Email	email with rank 1	emails		change_request_hco	email_1__v	N	N
configuration/entityTypes/HCO/attributes/Email/attributes/Email	email with rank 2		change_request_hco	email_2__v	N	N
configuration/entityTypes/HCO/attributes/Phone/attributes/Number	phone type TEL.FAX with best rank	phones		change_request_hco	fax_1__v	N	N
configuration/entityTypes/HCO/attributes/Phone/attributes/Number	phone type TEL.FAX with worst rank		change_request_hco	fax_2__v	N	N
configuration/entityTypes/HCO/attributes/StatusDetail		TODO - add new attribute		change_request_hco	hco_status__v	N	N		HCOStatus
configuration/entityTypes/HCO/attributes/TypeCode		typecode		change_request_hco	hco_type__v	N	N		HCOType
configuration/entityTypes/HCO/attributes/Phone/attributes/Number	phone type TEL.OFFICE with best rank	phones		change_request_hco	phone_1__v	N	N
configuration/entityTypes/HCO/attributes/Phone/attributes/Number	phone type TEL.OFFICE with worst rank		change_request_hco	phone_2__v	N	N
configuration/entityTypes/HCO/attributes/Phone/attributes/Number	phone type TEL.OFFICE with worst rank		change_request_hco	phone_3__v	N	N
configuration/entityTypes/HCO/attributes/Country		DCRRequest.country		change_request_hco	primary_country__v	N	N
configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty	elements from COT	specialties		change_request_hco	specialty_1__v	N	N
configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty		change_request_hco	specialty_10__v	N	N		Speciality
configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty		change_request_hco	specialty_2__v	N	N
configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty		change_request_hco	specialty_3__v	N	N
configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty		change_request_hco	specialty_4__v	N	N
configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty		change_request_hco	specialty_5__v	N	N
configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty		change_request_hco	specialty_6__v	N	N
configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty		change_request_hco	specialty_7__v	N	N
configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty		change_request_hco	specialty_8__v	N	N
configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty		change_request_hco	specialty_9__v	N	N
configuration/entityTypes/HCO/attributes/Website/attributes/WebsiteURL	first element	websiteURL		change_request_hco	URL_1__v	N	N
configuration/entityTypes/HCO/attributes/Website/attributes/WebsiteURL	N/A	N/A		change_request_hco	URL_2__v	N	N
HCP
N/A		Mongo Generated ID for this DCR	Kafka KEY		change_request_hcp	dcr_key	Y	Y	Customer's internal identifier for this request
Reltio Uri and Reltio Type	when insert new profile	entities.HCO.updateCrosswalk.type (Reltio) entities.HCO.updateCrosswalk.value (Reltio id) and refId.entityURI	concatenate Reltio:rvu44dm	change_request_hcp	entity_key	Y	Y	Customer's internal HCP identifier
configuration/entityTypes/HCP/attributes/Country		DCRRequest.country		change_request_hcp	primary_country__v	Y	Y
Crosswalks - VEEVA crosswalk	when update on VEEVA	entities.HCO.updateCrosswalk.type (VEEVA) entities.HCO.updateCrosswalk.value (VEEVA ID)		change_request_hcp	vid__v	N	Y
configuration/entityTypes/HCP/attributes/FirstName		firstName		change_request_hcp	first_name__v	Y	N
configuration/entityTypes/HCP/attributes/Middle		middleName		change_request_hcp	middle_name__v	N	N
configuration/entityTypes/HCP/attributes/LastName		lastName		change_request_hcp	last_name__v	Y	N
configuration/entityTypes/HCP/attributes/Nickname		TODO - add new attribute		change_request_hcp	nickname__v	N	N
configuration/entityTypes/HCP/attributes/Prefix		prefix		change_request_hcp	prefix__v	N	N		HCPPrefix
configuration/entityTypes/HCP/attributes/SuffixName		suffix		change_request_hcp	suffix__v	N	N
configuration/entityTypes/HCP/attributes/Title		title		change_request_hcp	professional_title__v	N	N		HCPProfessionalTitle
configuration/entityTypes/HCP/attributes/SubTypeCode		subTypeCode		change_request_hcp	hcp_type__v	Y	N		HCPType
configuration/entityTypes/HCP/attributes/StatusDetail		TODO - add new attribute		change_request_hcp	hcp_status__v	N	N		HCPStatus
configuration/entityTypes/HCP/attributes/AlternateName/attributes/FirstName		TODO - add new attribute		change_request_hcp	alternate_first_name__v	N	N
configuration/entityTypes/HCP/attributes/AlternateName/attributes/LastName		TODO - add new attribute		change_request_hcp	alternate_last_name__v	N	N
configuration/entityTypes/HCP/attributes/AlternateName/attributes/MiddleName		TODO - add new attribute		change_request_hcp	alternate_middle_name__v	N	N
??		TODO - add new attribute		change_request_hcp	family_full_name__v	N	N			TO BE CONFRIMED
configuration/entityTypes/HCP/attributes/DoB		birthYear		change_request_hcp	birth_year__v	N	N
configuration/entityTypes/HCP/attributes/Credential/attributes/Credential	by rank 1	TODO - add new attribute		change_request_hcp	credentials_1__v	N	N			TO BE CONFIRMED
configuration/entityTypes/HCP/attributes/Credential/attributes/Credential	2	TODO - add new attribute		change_request_hcp	credentials_2__v	N	N			In reltio there is attribute but not used
configuration/entityTypes/HCP/attributes/Credential/attributes/Credential	3	TODO - add new attribute		change_request_hcp	credentials_3__v	N	N			"uri": "configuration/entityTypes/HCP/attributes/Credential/attributes/Credential",
configuration/entityTypes/HCP/attributes/Credential/attributes/Credential	4	TODO - add new attribute		change_request_hcp	credentials_4__v	N	N			"lookupCode": "rdm/lookupTypes/Credential",
configuration/entityTypes/HCP/attributes/Credential/attributes/Credential	5	TODO - add new attribute		change_request_hcp	credentials_5__v	N	N		HCPCredentials	"skipInDataAccess": false
??		TODO - add new attribute		change_request_hcp	fellow__v	N	N		BooleanReference	TO BE CONFRIMED
configuration/entityTypes/HCP/attributes/Gender		gender		change_request_hcp	gender__v	N	N		HCPGender
?? Education ??		TODO - add new attribute		change_request_hcp	education_level__v	N	N		HCPEducationLevel	TO BE CONFRIMED
configuration/entityTypes/HCP/attributes/Education/attributes/SchoolName		TODO - add new attribute		change_request_hcp	grad_school__v	N	N
configuration/entityTypes/HCP/attributes/Education/attributes/YearOfGraduation		TODO - add new attribute		change_request_hcp	grad_year__v	N	N
??				change_request_hcp	hcp_focus_area_10__v	N	N			TO BE CONFRIMED
??				change_request_hcp	hcp_focus_area_1__v	N	N
??				change_request_hcp	hcp_focus_area_2__v	N	N
??				change_request_hcp	hcp_focus_area_3__v	N	N
??				change_request_hcp	hcp_focus_area_4__v	N	N
??				change_request_hcp	hcp_focus_area_5__v	N	N
??				change_request_hcp	hcp_focus_area_6__v	N	N
??				change_request_hcp	hcp_focus_area_7__v	N	N
??				change_request_hcp	hcp_focus_area_8__v	N	N
??				change_request_hcp	hcp_focus_area_9__v	N	N		HCPFocusArea
??				change_request_hcp	medical_degree_1__v	N	N			TO BE CONFRIMED
??				change_request_hcp	medical_degree_2__v	N	N		HCPMedicalDegree
configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialty	by rank from 1 to 100	specialties		change_request_hcp	specialty_1__v	Y	N
configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialty	specialties		change_request_hcp	specialty_10__v	N	N
configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialty	specialties		change_request_hcp	specialty_2__v	N	N
configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialty	specialties		change_request_hcp	specialty_3__v	N	N
configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialty	specialties		change_request_hcp	specialty_4__v	N	N
configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialty	specialties		change_request_hcp	specialty_5__v	N	N
configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialty	specialties		change_request_hcp	specialty_6__v	N	N
configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialty	specialties		change_request_hcp	specialty_7__v	N	N
configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialty	specialties		change_request_hcp	specialty_8__v	N	N
configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialty	specialties		change_request_hcp	specialty_9__v	N	N		Specialty
configuration/entityTypes/HCP/attributes/WebsiteURL		TODO - add new attribute		change_request_hcp	URL_1__v	N	N
ADDRESS
		Mongo Generated ID for this DCR	Kafka KEY		change_request_address	dcr_key	Y	Y	Customer's internal identifier for this request
Reltio Uri and Reltio Type	when insert new profile	entities.HCP OR HCO.updateCrosswalk.type (Reltio) entities.HCP OR HCO.updateCrosswalk.value (Reltio id) and refId.entityURI	concatenate Reltio:rvu44dm	change_request_address	entity_key	Y	Y	Customer's internal HCO/HCP identifier
attributes/Addresses/attributes/PfizerAddressID		address.refId		change_request_address	address_key	Y	Y	Customer's internal address identifier
attributes/Addresses/attributes/AddressLine1		addressLine1		change_request_address	address_line_1__v	Y	N
attributes/Addresses/attributes/AddressLine2		addressLine2		change_request_address	address_line_2__v	N	N
attributes/Addresses/attributes/AddressLine3		addressLine3		change_request_address	address_line_3__v	N	N
N/A		N/A	A	change_request_address	address_status__v	N	N		AddressStatus
attributes/Addresses/attributes/AddressType		addressType		change_request_address	address_type__v	Y	N		AddressType
attributes/Addresses/attributes/StateProvince		stateProvince		change_request_address	administrative_area__v	Y	N		AddressAdminArea
attributes/Addresses/attributes/Country		country		change_request_address	country__v	Y	N
attributes/Addresses/attributes/City		city		change_request_address	locality__v	Y	Y
attributes/Addresses/attributes/Zip5		zip		change_request_address	postal_code__v	Y	N
attributes/Addresses/attributes/Source/attributes/SourceName attributes/Addresses/attributes/Source/attributes/SourceAddressID	when VEEVA map VEEVA ID to	sourceAddressId		change_request_address	vid__v	N	Y
map from relationTypes/OtherHCOtoHCOAffiliations or relationTypes/ContactAffiliations		This will be HCP.ContactAffiliation or HCO.OtherHcoToHCO affiliation
		Mongo Generated ID for this DCR	Kafka KEY		change_request_parenthco	dcr_key	Y	Y	Customer's internal identifier for this request
		HCO.otherHCOAffiliations.relationUri or HCP.contactAffiliations.relationUri	(from Domain model) information about Reltio Relation ID	change_request_parenthco	parenthco_key	Y	Y	Customer's internal identifier for this relationship	RELATION ID
		KEY entity_key from HCP or HCO (start object)		change_request_parenthco	child_entity_key	Y	Y	Child Identifier in the HCO/HCP file	START OBJECT ID
endObject entity uri mapped to refId.EntityURITargetObjectId		KEY entity_key from HCP or HCO (end object, by affiliation)		change_request_parenthco	parent_entity_key	Y	Y	Parent identifier in the HCO file	END OBJECT ID
	changes in Domain model mapping	map Reltion.Source.SourceName - VEEVA map Relation.Source.SourceValue - VEEVA ID	add to Domain model map if relation is from VEEVA ID	change_request_parenthco	vid__v	N	Y
		start object entity type		change_request_parenthco	entity_type__v	Y	N
attributes/RelationType/attributes/PrimaryAffiliation		if is primary TODO - add new attribute to otherHcoToHCO		change_request_parenthco	is_primary_relationship__v	N	N		BooleanReference
		HCO_HCO or HCP_HCO		change_request_parenthco	hierarchy_type__v				RelationHierarchyType
attributes/RelationType/attributes/RelationshipDescription		type from affiliation based on ContactAffliation or OtherHCOToHCO affiliation	I think it will be 14-Emploted for HCP_HCO and 4-Manages for HCO_HCO but maybe we can map from affiliation.type	change_request_parenthco	relationship_type__v	Y	N		RelationType

Mongo collection

All DCRs initiated by the dcr-service-2 API and to be sent to Veeva will be stored in Mongo in new collection DCRRegistryVeeva. The idea is to gather all DCRs requested by the client through the day and schedule ‘SubmitVR’ process that will communicate with Veeva adapter.

Typical use case:

• Client requests 3 DCRs during the day
• SubmitVR contains the schedule that gathers all DCRs with NEW status created during the day and using VeevaAdapter to push requests to S3/SFTP.

Mapping Reltio canonical codes → Veeva source codes

There are a couple of steps performed to find out a mapping for canonical code from Reltio to source code understood by VOD. Below steps are performed (in this order) once a code is found.

Veeva Defaults

Configuration is stored in mdm-config-registry > config-hub/stage_apac/mdm-veeva-dcr-service/defaults

The purpose of these logic is to select one of possible multiple source codes on VOD end for a single code on Pfizer side (1:N). The other scenario is when there is no actual source code for a canonical code on VOD end (1:0), however this is usually covered by fallback code logic.

There are a couple of files, each containing source codes for a specific attribute. The ones related to HCO.Specialty and HCP.Specialty have logic which selects proper code.

• Usually there are constructed as a three column CSV: Country, Canonical Code, Source Code
• For specific Country we're looking for Canonical code and then we're sending Source code as it is (no trim required)
  • Examples: IN;SP.PD;PD → PD source code will be sent to VOD

RDM lookups with RegExp

The main logic which is used to find out proper source code for canonical code. We're using codes configured in RDM, however mongo collection LookupValues are used. For specific canonical code (code) we looking for sourceMappings with source = VOD. Often country is embedded within source code so we're applying regexpConfig (more in Veeva Fallback section) to extract specific source code for particular country.

Veeva Fallback

Configuration is stored in mdm-config-registry > config-hub/stage_apac/mdm-veeva-dcr-service/fallback

• Available for a couple of attributes:
  • hco-cot-facility-type.csv
    • COTFacilityType
  • hco-specialty.csv
    • COTSpecialty
  • hco-type-code.csv
    • HCOType
  • hcp-specialty.csv
    • HCPSpecialty
  • hcp-title.csv
    • HCPTitle
  • hcp-type-code.csv
    • HCPSubTypeCode
• Usually files are constructed as a one column CSV, however the logic for extracting source code may be different
• Source code is extracted using RegExp for each parameter. Check application.yml for this mdm-veeva-dcr-server component - mdm-inboud-services > mdm-veeva-dcr-service/src/main/resources/application.yml to find out proper line and extract code sent to VOD.
  • Example value for hco-specialty-type.csv: IN_?
  • Regexp value for HCP.specialty: regexpConfig > HCPSpecialty: ^COUNTRY_(.+)$
  • Source code sent to VOD for India country: "?" (only question mark without country prefix)

Triggers

Trigger action	Component	Action	Default time
REST call	mdm-veeva-dcr-service: POST /dcr → veevaDCRService.createChangeRequest(request)	Creates DCR and stores it in collection without actual send to Veeva.	API synchronous requests - realtime

Dependent components

Component	Usage
DCR Service 2	Main component with flow implementation
Hub Store	DCR and Entities Cache

---

Veeva: create DCR method (submitVR)

Page ID: 386796763 Page Link: /pages/viewpage.action?pageId=386796763

Description

Gather all stored DCR entities in DCRRegistryVeeva collection (status = NEW) and sends them via S3/SFTP to Veeva OpenData (VOD). This method triggers CSV/ZIP file creation and file placement in outbound directory. This method is triggered from cron which invokes VeevaDCRRequestSender.sendDCRs() from the Veeva DCR Service

Flow diagram

Steps

1. Receive the API request via scheduled trigger, usually every 24h (senderConfiguration.schedulerConfig.fixedDelay) at specific time of day (senderConfiguration.schedulerConfig.initDelay)
2. All DCR entities (VeevaVRDetails) with status NEW are being retrieved from DCRRegistryVeeva collection
3. Then VeevaCreateChangeRequest object is created which aggregates all CSV content which should be placed in actual CSV files.
   1. Each object contains only DCRs specific for country
   2. Each country has its own S3/SFTP directory structure as well as dedicated SFTP server instance
4. Once CSV files are created with header and content, they are packed into single ZIP file
5. Finally ZIP file is placed in outbound S3 directory
6. If file was placed
   1. successfuly - then VeevaChangeRequestACK status = SUCCESS
   2. otherwise - then VeevaChangeRequestACK status = FAILURE and process ends
7. Finally, status of VeevaVRDetails entity in DCRRegistryVeeva collection is updated and set to SENT_TO_VEEVA

Triggers

Trigger action	Component	Action	Default time
Timer (cron)	mdm-veeva-dcr-service: VeevaDCRRequestSender.sendDCRs()	Takes all unsent entities (status = NEW) from Veeva collection and actually puts file on S3/SFTP directory via veevaAdapter.createDCRs	Usually every 24h (senderConfiguration.schedulerConfig.fixedDelay) at specific time of day (senderConfiguration.schedulerConfig.initDelay)

Dependent components

Component	Usage
DCR Service 2	Main component with flow implementation
Hub Store	DCR and Entities Cache

---

Veeva: generate DCR Change Events (traceVR)

Page ID: 379329922 Page Link: /pages/viewpage.action?pageId=379329922

Description

The process is responsible for gathering DCR responses from Veeva OpenData (VOD). Responses are provided via CSV/ZIP files placed on S3/SFTP server in inbound directory which are specific for each country. During this process files should be retrieved, mapped from VOD to HUB DCR model and published to Kafka topic to be properly processed by DCR Service 2, Veeva: process DCR Change Events.

Flow diagram

Source: Lucid

Steps

1. Method is trigger via cron, usually every 24h (traceConfiguration.schedulerConfig.fixedDelay) at specific time of day (traceConfiguration.schedulerConfig.initDelay)
2. For each country, each inbound directory in scanned for ZIP files
3. Each ZIP files (<country>_DCR_Response_<Date>.zip) should be unpacked and processed. A bunch of CSV files should be extracted. Specifically:
   1. change_request_response.csv → it's a manifest file with general information in specific columns
      1. dcr_key → ID of DCR which was established during DCR request creation
      2. entity_key → ID of entity in Reltio, the same one we provided during DCR request creation
      3. entity_type → type of entity (HCO, HCP) which is being modified via this DCR
      4. resolution → has information whether DCR was accepted or rejected. Full list of values is below.

         1. resolution value	Description
            CHANGE_PENDING	This change is still processing and hasn't been resolved
            CHANGE_ACCEPTED	This change has been accepted without modification
            CHANGE_PARTIAL	This change has been accepted with additional changes made by the steward, or some parts of the change request have been rejected
            CHANGE_REJECTED	This change has been rejected in its entirety
            CHANGE_CANCELLED	This change has been cancelled

      5. change_request_type

         1. change_request_type value	Description
            ADD_REQUEST	whether DCR caused to create new profile in VOD with new vid__v  (Veeva id)
            CHANGE_REQUEST	just update of existing profile in VOD with existing and already known vid__v (Veeva id)

   2. change_request_hcp_response.csv - contains information about DCR related to HCP
   3. change_request_hco_response.csv - contains information about DCR related to HCO
   4. change_request_address_response.csv - contains information about DCR related to addresses which are related to specific HCP or HCO
   5. change_request_parenthco_response.csv - contains information about DCR which correspond to relations between HCP and HCO, and HCO and HCO
   6. File with log: <country>_DCR_Request_Job_Log.csv can be skipped. It does not contain any useful information to be processed automatically
4. For all DCR responses from VOD, we need to get corresponding DCR entity (VeevaVRDetails)from collection DCRRegistryVeeva should be selected.
5. In general, specific response files are not that important (VOD profiles updates will be ingested to HUB via ETL channel) however when new profiles are created (change_request_response.csv.change_request_type = ADD_REQUEST) we need to extract theirs Veeva ID.

   1. We need to deep dive into change_request_hcp_response.csv or change_request_hco_response.csv to find vid__v (Veeva ID) for specific dcr_key

   2. This new Veeva ID should be stored in VeevaDCREvent.vrDetails.veevaHCPIds

   3. It should be further used as a crosswalk value in Reltio:

      1. entities.HCO.updateCrosswalk.type (VEEVA)
      2. entities.HCO.updateCrosswalk.value (VEEVA ID)
6. Once data has been properly mapped from Veeva to HUB DCR model, new VeevaDCREvent entity should be created and published to dedicated Kafka topic $env-internal-veeva-dcr-change-events-in
   1. Please be advised, when the status of resolution is not final (CHANGE_ACCEPTED, CHANGE_REJECTED, CHANGE_CANCELLED, CHANGE_PARTIAL) we should not sent event to DCR-service-2
7. Then for each successfully processed DCR entity (VeevaVRDetails) in Mongo  DCRRegistryVeeva collection should be updated

   1. Veeva CSV: resolution	Mongo: DCRRegistryVeeva Entity: VeevaVRDetails.status: DCRRequestStatusDetails	Topic: $env-internal-veeva-dcr-change-events-in Event: VeevaDCREvent.vrDetails.vrStatus	Topic: $env-internal-veeva-dcr-change-events-in Event: VeevaDCREvent.vrDetails.vrStatusDetail
      CHANGE_PENDING	status should not be updated at all (stays as SENT)	do not send events to DCR-service-2	do not send events to DCR-service-2
      CHANGE_ACCEPTED	ACCEPTED	CLOSED	ACCEPTED
      CHANGE_PARTIAL	ACCEPTED	CLOSED	ACCEPTED resolutionNotes / veevaComment should contain more information what was rejected by VEEVA DS
      CHANGE_REJECTED	REJECTED	CLOSED	REJECTED
      CHANGE_CANCELLED	REJECTED	CLOSED	REJECTED

8. Once files are processed, ZIP file should be moved from inbound to archive directory

Triggers

Trigger action	Component	Action	Default time
IN Timer (cron)	mdm-veeva-dcr-service: VeevaDCRRequestTrace.traceDCRs()	get DCR responses from S3/SFTP directory, extract CSV files from ZIP file and publish events to kafka topic	every hour usually every 6h (traceConfiguration.schedulerConfig.fixedDelay) at specific time of day (traceConfiguration.schedulerConfig.initDelay)
OUT Events on Kafka Topic	mdm-veeva-dcr-service: VeevaDCRRequestTrace.traceDCRs() $env-internal-veeva-dcr-change-events-in	VeevaDCREvent event published to topic to be consumed by DCR Service 2	every hour usually every 6h (traceConfiguration.schedulerConfig.fixedDelay) at specific time of day (traceConfiguration.schedulerConfig.initDelay)

Dependent components

Component	Usage
DCR Service 2	Main component with flow implementation
Hub Store	DCR and Entities Cache

---