[ { "title": "DCR flows", "pageID": "415205424", "pageLink": "/display/GMDM/DCR+flows", "content": "
\n
\n
\n
\n

Overview

DCR (Data Change Request) process helps to improve existing data in source systems. Proposal for change is being created by source systems a as DCR object (sometimes also called VR - Validation Request) which is usually being routed by MDM HUB to DS (Data Stewards) either in Reltio or in Third party validators (OneKey, Veeva OpenData). Response is provided twofold:

  • response for specific DCR - metadata
  • profile data update as a direct effect of a DCR processing - payload


General DCR process flow

High level solution architecture for DCR flow


\"\"

Source: Lucid



\n
\n
\n
\n
\n
\n

Solution for OneKey (OK)

\"\"

\n
\n
\n
\n

Solution for Veeva OpenData (VOD)

\"\"

\n
\n
\n
\n
\n
\n

Architecture highlights

  • Actors involved: PforceRX, Reltio, HUB, OneKey
  • Key components: DCR Service 2 (second version) for AMER, EMEA, APAC, US tenants
  • Process details:
    • DCRs are created directly by PforceRx using DCR's HUB API
    • PforceRx checks for DCR status updates every 24h → finds out which DCRs has been updated (since last check 24h ago) and the pulls details from each one with /dcr/_status 
    • Integration with OneKey is realized by APIs - DCRs are created with /vr/submit and their status is verified every 8h with /vr/trace
    • Data profile updates (payload) are being delivered via CSV and S3 and ETLed (VOD batch) to Reltio with Deloitte's help
    • DCRRegistry & DCRRegistryVeeva collections are used in Mongo for tracking purposes




\n
\n
\n
\n

Architecture highlights

  • Actors involved: Data Stewards in Reltio, HUB, Veeva OpenData (VOD)
  • Key components: DCR Service 2 (second version) for AMER, EMEA, APAC, US tenants
  • Process details:
    • DCRs are created by Data Stewards (DSRs) in Reltio via Suggest / Send to 3rd Party Validation - input for DSRs is being provided by reports from PforceRx
    • Communication with Veeva via S3<>SFTP and synchronization GMTF jobs. DCRs are sent and received in batches every 24h 
    • DCRs metadata is being exchanged via multiple CSV files ZIPed
    • Data profile updates (payload) are being delivered via CSV and S3 and ETLed (VOD batch) to Reltio with Deloitte's help  
    • DCRRegistry & DCRRegistryONEKEY collections are used in Mongofor tracking purposes
\n
\n
\n
\n
\n
\n

Solution for IQVIA Highlander (HL) 

\"\"


\n
\n
\n
\n

Solution for OneKey on GBLUS - sources ICEU, Engage, GRV

\n
\n
\n
\n
\n
\n

Architecture highlights

  • Actors involved: Veeva on behalf of PforceRX, Reltio, HUB, IQVIA wrapper
  • Key components: DCR Service (first version) for GBLUS tenant
  • Process details:
    • DCRs are created by sending CSV requests by Veeva - based on information acquired from PforceRx
    • Integration HUB <> Veeva → via files and S3<>SFTP. HUB confirms DCR creation by returning file reports back to Veeva
    • Integration HUB <> IQVIA wrapper → via files and S3
    • HUB is responsible for translation of Veeva DCR CSV format to IQVIA CSV wrapper which then creates DCR in Reltio
    • Data Stewards approve or reject the DCRs in Reltio which updates data profiles accordingly. 
    • PforceRx receives update about changes in Reltio
    • DCRRequest collection is used in Mongo for tracking purposes
\n
\n
\n
\n

Architecture highlights (draft)

  • Actors involved: HUB, IQVIA wrapper
  • Key components: DCR Service (first version) for GBLUS tenant
  • Process details:
    • POST events from sources are captured - some of them are translated to direct DCRs, some of them are gathered and then pushed via flat files to be transformed into DCRs to OneKey

 


\n
\n
\n
" }, { "title": "DCR generation process (China DCR)", "pageID": "164470008", "pageLink": "/pages/viewpage.action?pageId=164470008", "content": "

The gateway supports following DCR types:


DCR generation processes are handled in two steps:

  1. During HCP modification – if initial activation criteria are met, then a DCR request is generated and published to KAFKA <env>-gw-dcr-requests topic.
  2. In the next step, the internal Camel route DCRServiceRoute reads requests generated from the topic and processes as follows:
    1. checks if the time specified by delayPrcInSeconds elapsed since request generation – it makes sure that Reltio batch match process has finished and newly inserted profiles merge with the existing ones.
    2. checks if an entity, that caused DCR generation, still exists;
    3. checks full activation criteria (table below) on the latest state of the target entity, if criteria are not met then the request is closed
    4. creates DCR in Reltio
    5. updates external info
    6. creates PfizerDataChangeRequest entity in Reltio for tracking and exporting purposes.
  3. Created DCRs are exported by the Informatica ETL process managed by IQIVIA
  4. DCR applying process (reject/approve actions) are executed through MDM HUB DCR response API executed by the external app manged by MDE team.


The table below presents DCR activation criteria handled by system.

Table 9. DCR activation criteria





Rule

NewHCP

MultiAffiliation

NewHCOL2

NewHCOL1

Country in

CN

CN

CN

CN

Source in

GRV

GRV, MDE, FACE, EVR, CN3RDPARTY

GRV, FACE, CN3RDPARTY

GRV, FACE, CN3RDPARTY

ValidationStatus in

pending, partial-validated

or, if merged:

OV: notvalidated, GRV nonOV: pending/partial-validated

validated, pending

validated, pending

validated, pending

SpeakerStatus in

enabled, null

enabled, null

enabled, null

enabled, null

Workplaces count


>1



Hospital found

true

true

false

true

Department found

true

true


false

Similar DCR created in the past

false

false

false

false


Update: December 2021

\"\"

" }, { "title": "HL DCR [Decommissioned April 2025]", "pageID": "164470085", "pageLink": "/pages/viewpage.action?pageId=164470085", "content": "

Contacts

VendorContact
PforceRX

DL-PForceRx-SUPPORT@pfizer.com

IQVIA (DCR Wrapper)Pfizer-MDM-Support@iqvia.com 


As a part of Highlander project, the DCR processing flow was created which realizes following scenarios:

  1. Update HCP account details i.e. specialty, address, name (different sources of elements),
  2. Add new HCP account with primary affiliation to an existing organization,
  3. Add new HCP account with a new business account,
  4. Update HCP and add affiliation to a new HCO,
  5. Update HCP account details and remove existing details i.e. birth date, national id, …,
  6. Update HCP account and add new non primary affiliation to an existing organization,
  7. Update HCP account and add new primary affiliation to an existing organization,
  8. Update HCP account inactivate primary affiliation. Person account has more than 1 affiliation,
  9. Update HCP account inactivate non primary affiliation. Person account has more than 1 affiliation,
  10. Inactivate HCP account,
  11. Update HCP and add a private address,
  12. Update HCP and update existing private address,
  13. Update HCP and inactivate a private address,
  14. Update HCO details i.e. address, name (different sources of elements),
  15. Add new HCO account,
  16. Update HCO and remove details,
  17. Inactivate HCO account,
  18. Update HCO address,
  19. Update HCO and add new address,
  20. Update HCO and inactivate address,
  21. Update HCP's existing affiliation.


Above cases has been aggregated into six generic types in internal HUB model:

  1. NEW_HCP_GENERIC - represents cases when the new HCP object is created with or without affiliation to HCO,
  2. UPDATE_HCP_GENERIC - aggregates cases when the existing HCP object is changed,
  3. DELETE_HCP_GENERIC - represents the case when HCP is deactivating,
  4. NEW_HCO_GENERIC - aggregates scenarios when new HCO object is created with or without affiliations to parent HCO,
  5. UPDATE_HCO_GENERIC - represents cases when existing HCO object is changing,
  6. DELETE_HCO_GENERIC - represents the case when HCO is deactivating.


General Process Overview

\"\"


Process steps:

  1. Veeva uploads DCR request file to FTP location,
  2. PforceRx Channel component downloads the DCR request file,
  3. PforceRx Channel validates and maps each DCR requests to internal model,
  4. PforceRx Channel sends the request to DCR Service,
  5. DCR Service process the request: validating, enriching and mapping to Iqvia DCR Wrapper,
  6. PforceRx Channel prepares the report file containing technical status of DCR processing - at this time, report will contain only requests which don't pass the validation,
  7. Scheduled process in DCR Service, prepares the Wrapper requests file and uploads this to S3 location.
  8. DCR Wrapper processes the file: creating DCRs in Reltio or rejecting the request due to errors. After that the response file is published to s3 location,
  9. DCR Service downloads the response and updates DCRs status,
  10. Scheduled process in PforceRx Channel gets DCR requests and prepares next technical report - at this time the report has technical status which comes from DCR Wrappper,
  11. DCRs that was created by DCR Wrapper are reviewed by Data Stewards. DCR can be accepted or rejected,
  12. After accepting or rejecting DCR, Reltio publishes the message about this event,
  13. DCR Service consumes the message and updates DCR status,
  14. PforceRx Channel gets DCR data to prepare a response file. The response file contains the final status of DCRs processing in Reltio.


Veeva DCR request file specification

The specification is available at following location:

https://pfizer-my.sharepoint.com/:x:/r/personal/chinj2_pfizer_com/Documents/Mig%20In-Prog/Highlander/PMO/09%20Integration/LATAM%20Reltio%20DCR/DCR_Reltio_T144_Field_Mapping_Reltio.xlsx


DCR Wrapper request file specification

The specification is available at following link:

https://pfizer.sharepoint.com/:x:/r/sites/HLDCR/Shared%20Documents/ReltioCloudMDM_LATAM_Highlander_DCR_DID_PFIZER__DEVMapping_v2.1.xlsx





" }, { "title": "OK DCR flows (GBLUS)", "pageID": "164469877", "pageLink": "/pages/viewpage.action?pageId=164469877", "content": "

Description

The process is responsible for creating DCRs in Reltio and starting Change Requests Workflow for singleton entities created in Reltio. During this process, the communication to IQVIA OneKey VR API is established.  SubmitVR operation is executed to create a new Validation Request. The TraceVR operation is executed to check the status of the VR in OneKey. All DCRs are saved in the dedicated collection in HUB Mongo DB, required to gather metadata and trace the changes for each DCR request. Some changes can be suggested by the DS using "Suggest" operation in Reltio and "Send to Third Party Validation" button, the process "Data Steward OK Validation Request" is processing these changes and sends them to the OneKey service. 

The process is divided into 4 sections:

  1. Submit Validation Request
  2. Trace Validation Request
  3. Data Steward Response
  4. Data Steward OK Validation Request

The below diagram presents an overview of the entire process. Detailed descriptions are available in the separated subpages.

Flow diagram

\"\"

Model diagram

\"\"

Steps

Triggers

Described in the separated sub-pages for each process.

Dependent components

Described in the separated sub-pages for each process.

" }, { "title": "Data Steward OK Validation Request", "pageID": "172306908", "pageLink": "/display/GMDM/Data+Steward+OK+Validation+Request", "content": "

Description

The process the DS suggested changes based on the Change Request events received from Reltio(publishing) that are marked with the ThirdPartyValidation flag. The "suggested" changes are retrieved using the "preview" method and send to IQVIA OneKey or Veeva OpenData for validation. After successful submitVR response HUB is closing/rejecting the existing DCR in Reltio and additionally creates a new DCR object with relation to the entity in Reltio for tracking and status purposes. 

Because of the ONEKEY interface limitation, removal of attributes is send to IQVIA as a comment.

Flow diagram

\"\"


Steps


 ONEKEY Comparator (suggested changes)

HCP

Reltio AttributeONEKEY attributemandatory typeattribute type
FirstName
individual.firstNameoptionalsimple value
LastNameindividual.lastNamemandatorysimple value
Country
isoCod2
mandatorysimple value
Genderindividual.genderCodeoptionalsimple lookup
Prefixindividual.prefixNameCodeoptionalsimple lookup
Titleindividual.titleCodeoptionalsimple lookup
MiddleNameindividual.middleNameoptionalsimple value
YoBindividual.birthYearoptionalsimple value
Dobindividual.birthDayoptionalsimple value
TypeCodeindividual.typeCodeoptionalsimple lookup
PreferredLanguageindividual.languageEidoptionalsimple value
WebsiteURL
individual.websiteoptionalsimple value

Identifier value 1

individial.externalId1optionalsimple value

Identifier value 2

individial.externalId2optionalsimple value
Addresses[]

address.country

address.city

address.addressLine1

address.addressLine2

address.Zip5

mandatorycomplex (nested)
Specialities[]
individual.speciality1 / 2 / 3optionalcomplex (nested)
Phone[]
individual.phoneoptionalcomplex (nested)
Email[]
individual.emailoptionalcomplex (nested)
Contact Affiliations[]

workplace.usualName

workplace.officialName

workplace.workplaceEid

optionalContact Affiliation
ONEKEY crosswalk
individual.individualEid
mandatoryID

HCO

Reltio AttributeONEKEY attributemandatory typeattribute type
Name

workplace.usualName

workplace.officialName

optionalsimple value
Country
isoCod2
mandatorysimple value
OtherNames.Name
workplace.usualName2optionalcomplex (nested)
TypeCodeworkplace.typeCodeoptionalsimple lookup
WebisteWebsiteURL
workplace.websiteoptionalcomplex (nested)
Addresses[]

address.country

address.city

address.addressLine1

address.addressLine2

address.Zip5

mandatorycomplex (nested)
Specialities[]
workplace.speciality1 / 2 / 3optionalcomplex (nested)
Phone[] (!FAX)
workplace.telephoneoptionalcomplex (nested)
Phone[] (FAX)
workplace.faxoptionalcomplex (nested)
Email[]
workplace.emailoptionalcomplex (nested)
ONEKEY crosswalk
workplace.workplaceEid
mandatoryID



Triggers

Trigger actionComponentActionDefault time
IN Events incoming mdm-onekey-dcr-service:ChangeRequestStreamprocess publisher full change request events in the stream that contain ThirdPartyValidation flagrealtime: events stream processing 

Dependent components

ComponentUsage
OK DCR ServiceMain component with flow implementation
Veeva DCR ServiceMain component with flow implementation
PublisherEvents publisher generates incoming events
Hub StoreDCR and Entities Cache 
" }, { "title": "Data Steward Response", "pageID": "164469841", "pageLink": "/display/GMDM/Data+Steward+Response", "content": "

Description

The process updates the DCR's based on the Change Request events received from Reltio(publishing). Based on the Data Steward decision the state attribute contains relevant information to update DCR status.

Flow diagram


\"\"

Steps

Triggers

Trigger actionComponentActionDefault time
IN Events incoming 

mdm-onekey-dcr-service:OneKeyResponseStream

mdm-veeva-dcr-service:veevaResponseStream

process publisher full change request events in streamrealtime: events stream processing 

Dependent components

ComponentUsage
OK DCR ServiceMain component with flow implementation
Veeva DCR ServiceMain component with flow implementation
PublisherEvents publisher generates incoming events
Hub StoreDCR and Entities Cache 
" }, { "title": "Submit Validation Request", "pageID": "164469875", "pageLink": "/display/GMDM/Submit+Validation+Request", "content": "

Description

The process of submitting new validation requests to the OneKey service based on the Reltio change events aggregated in time windows. During this process, new DCRs are created in Reltio.

Flow diagram


\"\"

Steps


Triggers

Trigger actionComponentActionDefault time
IN Events incoming mdm-onekey-dcr-service:OneKeyStreamprocess publisher simple events in streamevents stream processing with 4h time window events aggregation
OUT API requestone-key-client:OneKeyIntegrationService.submitValidationsubmit VR request to OneKeyinvokes API request for each accepted event

Dependent components

ComponentUsage
OK DCR ServiceMain component with flow implementation
PublisherEvents publisher generates incoming events
ManagerReltio Adapter for getMatches and created operations
OneKey AdapterSubmits Validation Request
Hub StoreDCR and Entities Cache 

Mappings

Reltio → OK mapping file: onkey_mappings.xlsx

OK mandatory / required fields: VR - Business Fields Requirements(Pfizer).xlsx

OneKey Documentation

\"\"




" }, { "title": "Trace Validation Request", "pageID": "164469983", "pageLink": "/display/GMDM/Trace+Validation+Request", "content": "

Description

The process of tracing the VR changes based on the OneKey VR changes. During this process HUB, DCR Cache is triggered every <T> hour for SENT DCR's and check VR status using OneKey web service. After verification DCR is updated in Reltio or a new Workflow is started in Reltio for the Data Steward manual validation. 

Flow diagram


\"\"


Steps



Triggers

Trigger actionComponentActionDefault time
IN Timer (cron)mdm-onekey-dcr-service:TraceVRServicequery mongo to get all SENT DCR's related to OK_VR processevery <T> hour
OUT API requestone-key-client:OneKeyIntegrationService.traceValidationtrace VR request to OneKeyinvokes API request for each DCR

Dependent components

ComponentUsage
OK DCR ServiceMain component with flow implementation
ManagerReltio Adapter for GET /changeRequests and POST /workflow/_initiate operations 
OneKey AdapterTraceValidation Request
Hub StoreDCR and Entities Cache 



" }, { "title": "PforceRx DCR flows", "pageID": "209949183", "pageLink": "/display/GMDM/PforceRx+DCR+flows", "content": "

Description

MDM HUB exposes Rest API to create and check the status of DCR. The process is responsible for creating DCRs in Reltio and starting Change Requests Workflow DCRs created in Reltio or creating the DCRs (submitVR operation) in ONEKEY. DCR requests can be routed to an external MDM HUB instance handling the requested country. The action is transparent to the caller. During this process, the communication to IQVIA OneKey VR API / Reltio API is established. The routing decision depends on the market, operation type, or changed profile attributes.

Reltio API:  createEntity (with ChangeReqest) operation is executed to create a completely new entity in the new Change Request in Reltio. attributesUpdate (with ChageRequest) operation is executed after calculation of the specific changes on complex or simple attributes on existing entity - this also creates a new Change Request.  Start Workflow operation is requested at the end, this starts the Wrofklow for the DCR in Reltio so the change requests are started in the Reltio Inbox for Data Steward review.

IQVIA API: SubmitVR operation is executed to create a new Validation Request. The TraceVR operation is executed to check the status of the VR in OneKey.

All DCRs are saved in the dedicated collection in HUB Mongo DB, required to gather metadata and trace the changes for each DCR request. The DCR statuses are updated by consuming events generated by Reltio or periodic query action of open DCRs in OneKey

The Data Steward can decide to route a DCR to IQVIA as well - some changes can be suggested by the DS using the "Suggest" operation in Reltio and "Send to Third Party Validation" button, the process "Data Steward OK Validation Request" is processing these changes and sends them to the OneKey service. 

The below diagram presents an overview of the entire process. Detailed descriptions are available in the separated subpages.

API doc URL: https://api-emea-nprod-gbl-mdm-hub.pfizer.com/api-dcr-spec-emea-dev/swagger-ui/index.html

Flow diagram

DCR Service High-Level Architecture

\"\"

DCR HUB Logical Architecture

\"\"


Model diagram


\"\"

Flows:

Triggers

Described in the separated sub-pages for each process.

Dependent components

Described in the separated sub-pages for each process.


" }, { "title": "Create DCR", "pageID": "209949185", "pageLink": "/display/GMDM/Create+DCR", "content": "

Description

The process creates change requests received from PforceRx Client and sends the DCR to the specified target service - Reltio, OneKey or Veeva OpenData (VOD). DCR is created in the system and then processed by the data stewards. The status is asynchronously updated by the HUB processes, Client represents the DCR using a unique extDCRRequestId value. Using this value Client can check the status of the DCR (Get DCR status). 

Flow diagram

\"\"

Source: Lucid

\"\"

Source: Lucid


DCR Service component perspective


Steps


  1. Clients execute the API POST /dcr request
  2. Kong receives requests and handles authentication
  3. If the authentication succeeds the request is forwarded to the dcr-service-2 component,
  4. DCR Service checks permissions to call this operation and the correctness of the request, then the flow is started and the following steps are executed:
    1. Parse and validate the dcr request. The validation logic checks the following: 
      1. Check if the list of DCRRequests contains unique extDCRRequestId.
        1. Requests that are duplicate will be rejected with the error message - "Found duplicated request(s)"
      2. For each DCRRequest in the input list execute the following checks:
        1. Users can define the following number of entities in the Request:
          1. at least one entity has to be defined, otherwise, the request will be rejected with an error message - "No entities found in the request"
          2. single HCP
          3. singe HCO
          4. singe HCP with single HCO
          5. two HCOs
        2. Check if the main reference objects exist in Reltio for update and delete action
          1. HCP.refId or HCO.refId, user have to specify one of:
            1. CrosswalkTargetObjectId - then the entity is retrieved from Reltio using get entity by crosswalk operation
            2. EntityURITargetObjectId - then the entity is retrieved from Reltio using get entity by uri operation
            3. PfizerCustomerIdTargetObjectId - then the entity is retrieved from Reltio using search operation by the PfizerGlobalCustomerID
        3. Attributes validation:
          1. Simple attributes - like firstName/lastName e.t.c
            1. for update action on the main object:
              1. if the input parameter is defined with an empty value - "" - this will result in the removal of the target attribute
              2. if the input parameter is defined with a non-empty value - this will result in the update of the target attribute
          2. Nested attributes - like Specialties/Addresses e.t.c
            1. for each attribute, the user has to define the refId to uniquely identify the attribute
              1. For action "update" - if the refId is not found in the target object request will be rejected with a detailed error message 
              2. For action "insert" - the refId is not required - new reference attribute will be added to the target object
        4. Changes validation:
          1. If the validation detected 0 changes (during comparison of applying changes and the target entity) -  the request is rejected with an error message - "No changes detected"
    2. Evaluate dcr service (based on the decision table config)
      1. The following decision table is defined to choose the target service

        1. LIST OF the following combination of attributes:

          attributedescription
          userName
          		the user name that executes the request
          sourceName
          		the source name of the Main object
          country
          		the county defined in the request
          operationType

          the operation type for the Main object

          { insert, update, delete }
          affectedAttributes
          		the list of attributes that the user is changing
          affectedObjects
          { HCP, HCO, HCP_HCO }

          RESULT →  TargetType {Reltio, OneKey, Veeva}

        2. Each attribute in the configuration is optional. 

        3. The decision table is making the validation based on the input request and the main object- the main object is HCP, if the HCP is empty then the decision table is checking HCO. 
        4. The result of the decision table is the TargetType, the routing to the Reltio MDM system, OneKey or Veeva service. 
    3. Execute target service (reltio/onekey/veeva)
      1. Reltio: create DCR method - direct
      2. OneKey: create DCR method (submitVR) - direct
      3. Veeva: create DCR method (storeVR)
    4. Create DCR in Reltio and save DCR in DCR Registry 
      • If the submission is successful then: 
        • DCR entity is created in Reltio and the relation between the processed entity and the DCR entity
          • Reltio source name (crosswalk.type): DCR
          • Reltio relation type: HCPtoDCR or HCOtoDCR (depending on the object type)
            • for "create" and "delete" operation the Relation have to be created between objects
            • if this is just the "insert" operation the Relation will be created after the acceptance of the Change Request in Reltio - Reltio: process DCR Change Events

          • DCR entity attributes once sent to OneKey

            DCR entity attributes

            Mapping

            DCRIDextDCRRequestId
            EntityURIthe processed entity URI
            VRStatus"OPEN"
            VRStatusDetail"SENT_TO_OK"
            CreatedByMDM HUB
            SentDatecurrent time

            CreateDate

            current time

            CloseDate

            if REJECTED | ACCEPTED -> current time

            dcrType

            evaluate based on config:

            dcrTypeRules:
              - type: CR0
                size: 1
                action: insert
                entity: com.pfizer.mdm.api.dcr2.HCP

            \"\"

          • DCR entity attributes once sent to Veeva

            DCR entity attributes

            Mapping

            DCRIDextDCRRequestId
            EntityURIthe processed entity URI
            VRStatus"OPEN"
            VRStatusDetail"SENT_TO_VEEVA"
            CreatedByMDM HUB
            SentDatecurrent time

            CreateDate

            current time

            CloseDate

            if REJECTED | ACCEPTED -> current time

            dcrType

            evaluate based on config:

            dcrTypeRules:
              - type: CR0
                size: 1
                action: insert
                entity: com.pfizer.mdm.api.dcr2.HCP

            \"\"

          • DCR entity attributes once sent to Reltio → action is passed to DS and workflow is started. 

            DCR entity attributes

            Mapping

            DCRIDextDCRRequestId
            EntityURIthe processed entity URI
            VRStatus"OPEN"
            VRStatusDetail"DS_ACTION_REQUIRED "
            CreatedByMDM HUB
            SentDatecurrent time

            CreateDate

            current time

            CloseDate

            if REJECTED | ACCEPTED -> current time

            dcrType

            evaluate based on config:

            dcrTypeRules:
              - type: CR0
                size: 1
                action: insert
                entity: com.pfizer.mdm.api.dcr2.HCP

            \"\"

        • Mongo Update: DCRRequest.status is updated to SENT with OneKey or Veeva request and response details or DS_ACTION_REQURIED with all Reltio details
      • Otherwise FAILED status is recorded in DCRRequest with a detailed error message.
        • Mongo Update:  DCRRequest.status is updated to FAILED with all required attributes, request, and exception response details 
    5. Initialize Workflow in Reltio (only requests that TargetType is Reltio)

      1. POST /workflow/_initiate operation is invoked to init new Workflow in Reltio

        Workflow attributes

        Mapping

        changeRequest.uriChangeRequest Reltio URI
        changeRequest.changesEntity URI
    6. Then Auto close logic is invoked to evaluate whether DCR request meets conditions to be auto accepted or auto rejected. Logic is based on decision table PreCloseConfig. If DCRRequest.country is contained in PreCloseConfig.acceptCountries or PreCloseConfig.rejectCountries then DCR is accepted or rejected respectively. 
    7. return DCRResponse to Client - During the flow, DCRRespone may be returned to Client with the specific errorCode or requestStatus. The description for all response codes is presented on this page: Get DCR status

Triggers

Trigger actionComponentActionDefault time
REST callDCR Service: POST /dcrcreate DCRs in the Reltio, OneKey or Veeva systemAPI synchronous requests - realtime


Dependent components

ComponentUsage
DCR ServiceMain component with flow implementation
OK DCR ServiceOneKey Adapter - API operations
Veeva DCR ServiceVeeva Adapter - API operations and S3/SFTP communication 
ManagerReltio Adapter - API operations
Hub StoreDCR and Entities Cache 
" }, { "title": "DCR state change", "pageID": "218438617", "pageLink": "/display/GMDM/DCR+state+change", "content": "

Description

The following diagram represents the DCR state changes. DCR object stat is saved in HUB and in Reltio DCR entity object. The state of the DCR is changed based on the Reltio/IQVIA/Veeva Data Steward action.

Flow diagram

\"\"

Steps

  1. DCR is created (OPEN)  - Create DCR
    1. DCR is sent to Reltio, OneKey or Veeva
      1. When sent to Reltio
        1. Pre Close logic is invoked to auto accept (PRE_ACCEPT) or auto reject (PRE_REJECT) DCR
        2. Reltio Data Steward process the DCR - Reltio: process DCR Change Events
      2. OneKey Data Steward process the DCR - OneKey: process DCR Change Events
      3. Veeva Data Steward process the DCR - Veeva: process DCR Change Events


Data Steward DCR status change perspective

\"\"

Transaction Log

There are the following main assumptions regarding the transaction log in DCR service: 


Log appenders:


Triggers

Trigger action

Component

Action

Default time

REST callDCR Service: POST /dcrcreate DCRs in the Reltio system or in OneKeyAPI synchronous requests - realtime
IN Events incoming dcr-service-2:DCRReltioResponseStreamprocess publisher full change request events in the streamrealtime: events stream processing 
IN Events incoming dcr-service-2:DCROneKeyResponseStreamprocess publisher full change request events in the streamrealtime: events stream processing 
IN Events incoming dcr-service-2:DCRVeevaResponseStreamprocess publisher full change request events in the streamrealtime: events stream processing 


Dependent components

Component

Usage

DCR ServiceMain component with flow implementation
OK DCR ServiceOneKey Adapter  - API operations
Veeva DCR ServiceVeeva Adapter  - API operations
ManagerReltio Adapter  - API operations
Hub StoreDCR and Entities Cache 
" }, { "title": "Get DCR status", "pageID": "209949187", "pageLink": "/display/GMDM/Get+DCR+status", "content": "

Description

The client creates DCRs in Reltio, OneKey or Veeva OpenData using the Create DCR operation. The status is then asynchronously updated in the DCR Registry. The operation retrieves the current status of the DCRs that the updated date is between 'updateFrom' and 'updateTo' input parameters. PforceRx first asks what DCRs have been changed since last time they checked (usually 24h) and then iterate for each DCR they get detailed info.

Flow diagram

\"\",

\"\"

Source: Lucid



Dependent flows:
  1. The DCRRegistry is enriched by the DCR events that are generated by Reltio - the flow description is here - Reltio: process DCR Change Events
  2. The DCRRegistry is enriched by the DCR events generated in OneKey DCR service component - after submitVR operation is invoked to ONEKEY, each DCR is traced asynchronously in this process - OneKey: process DCR Change Events
  3. The DCRRegistry is enriched by the DCR events generated in Veeva OpenData DCR service component - after submitVR operation is invoked to VEEVA, each DCR is traced asynchronously in this process - Veeva: process DCR Change Events

Steps

Status

There are the following request statuses that users may receive during Create DCR operation or during checking the updated status using GET /dcr/_status operation described below:

RequestStatusDCRStatus Internal Cache statusDescription
REQUEST_ACCEPTEDCREATEDSENT_TO_OKDCR was sent to the ONEKEY system for validation and pending the processing by Data Steward in the system
REQUEST_ACCEPTEDCREATEDSENT_TO_VEEVADCR was sent to the VEEVA system for validation and pending the processing by Data Steward in the system
REQUEST_ACCEPTEDCREATEDDS_ACTION_REQUIREDDCR is pending Data Steward validation in Reltio, waiting for approval or rejection
REQUEST_ACCEPTEDCREATEDOK_NOT_FOUNDUsed when ONEKEY profile was not found after X retries
REQUEST_ACCEPTEDCREATEDVEEVA_NOT_FOUNDUsed when VEEVA profile was not found after X retries
REQUEST_ACCEPTEDCREATEDWAITING_FOR_ETL_DATA_LOADUsed when waiting for actual data profile load from 3rd Party to appear in Reltio
REQUEST_ACCEPTEDACCEPTEDACCEPTEDData Steward accepted the DCR, changes were applied
REQUEST_ACCEPTEDACCEPTEDPRE_ACCEPTEDPreClose logic was invoked and automatically accepted DCR according to decision table in PreCloseConfig
REQUEST_REJECTEDREJECTED REJECTEDData Steward rejected the changes presented in the Change Request
REQUEST_REJECTEDREJECTED PRE_REJECTEDPreClose logic was invoked and automatically rejected DCR according to decision table in PreCloseConfig
REQUEST_FAILED-FAILEDDCR requests failed due to: validation error/ unexpected error e.t.d - details in the errorCode and errorMessage
Error codes:

There are the following classes of exception that users may receive during Create DCR operation:

ClasserrorCodeDescriptionHTTP code
1DUPLICATE_REQUESTrequest rejected - extDCRRequestId  is registered - this is a duplicate request403
2NO_CHANGES_DETECTEDentities are the same (request is the same) - no changes400
3VALIDATION_ERRORref object does not exist (not able to find HCP/HCO target object404
3VALIDATION_ERRORref attribute does not exist - not able to find nested attribute in the target object400
3VALIDATION_ERRORwrong number of HCP/HCO entities in the input request400


  1. Clients execute the API GET/dcr/_status request
  2. Kong receives requests and handles authentication
  3. If the authentication succeeds the request is forwarded to the dcr-service-2 component,
  4. DCR Service checks permissions to call this operation and the correctness of the request, then the flow is started and the following steps are executed
    1. Query on mongo is executed to get all DCRs matching input parameters:
      1. updateFrom (date-time) - DCR last update from - DCRRequestDetails.status.changeDate
      2. updateTo (date-time) - DCR last update to - DCRRequestDetails.status.changeDate
      3. limit (int) the maximum number of results returned through API - the recommended value is 25. The max value for a single request is 50.
      4. offset(int) - result offset - the parameter used to query through results that exceeded the limit. 
    2. Resulted values are aggregated and returned to the Client.
    3. The client receives the List<DCRResposne> body.

Triggers

Trigger action

Component

Action

Default time

REST callDCR Service: GET/dcr/_statusget status of created DCRs. Limit the results using query parameters like dates and offsetAPI synchronous requests - realtime


Dependent components

Component

Usage

DCR ServiceMain component with flow implementation
Hub StoreDCR and Entities Cache 
" }, { "title": "OneKey: create DCR method (submitVR) - direct", "pageID": "209949294", "pageLink": "/display/GMDM/OneKey%3A+create+DCR+method+%28submitVR%29+-+direct", "content": "

Description

Rest API method exposed in the OK DCR Service component responsible for submitting the VR to OneKey

Flow diagram

\"\"

Steps


  1. Receive the API request
  2. Validate - check if the onekey crosswalk exists once there is an update on the profile, otherwise reject the request
  3. The DCR is mapped to OK VR Request and it's submitted using API REST method POST /vr/submit. (mapping described below)
    1. If the submission is successful then:
      • DCRRequesti updated to SENT_TO_OK with OK request and response details. DCRRegistryONEKEY collection in saved for tracing purposes. The process that reads and check ONEKEY VRs is described here: OneKey: generate DCR Change Events (traceVR)
    2. Otherwise FAILED status is recorded and the response is returned with an OK error response

Mapping


VR - Business Fields Requirements_UK.xlsx - file that contains VR UK requirements and mapping to IQVIA model


HUB

ONEKEY

attributesattributescodes
mandatoryattributesvalues

HCO













YentityTypeWORKPLACE





Yvalidation.clientRequestIdHUB_GENERATED_ID





Yvalidation.processQ





Yvalidation.requestDate1970-01-01T00:00Z





Yvalidation.callDate1970-01-01T00:00Z
attributes



Yvalidation.requestProcessI

extDCRComment



validation.requestComment









country


YisoCod2

















reference EntitycrosswalkONEKEY

workplace.workplaceEid









name



workplace.usualName






workplace.officialName

otherHCOAffiliationsparentUsualName


workplace.parentUsualName

subTypeCode

COTFacilityType

(TET.W.*)



workplace.typeCode

typeCodeno value in PFORCERX

HCOSubType

(LEX.W.*)



workplace.activityLocationCode

addresses







sourceAddressId


N/A


addressType


N/A


addressLine1


address.longLabel


addressLine2


address.longLabel2


addressLine3


N/A


stateProvince

AddressState

(DPT.W.*)



address.countyCode


city

Yaddress.city


zip


address.longPostalCode


country

Yaddress.country


rank


get address with rank=1 

emails







type


N/A


email


workplace.email


rank


get email with rank=1 

otherHCOAffiliations







type


N/A


rank


get affiliation with rank=1 

reference EntityotherHCOAffiliations reference entity onekeyID ONEKEY

workplace.parentWorkplaceEid

phones







typecontains FAX





number


workplace.telephone


rank


get phone with rank=1 










typenot contains FAX





number


workplace.fax


rank


get phone with rank=1 

HCP













YentityTypeACTIVITY





Yvalidation.clientRequestIdHUB_GENERATED_ID





Yvalidation.processQ





Yvalidation.requestDate1970-01-01T00:00Z





Yvalidation.callDate1970-01-01T00:00Z
attributes



Yvalidation.requestProcessI

extDCRComment



validation.requestComment









country


YisoCod2

















reference EntitycrosswalkONEKEY

individual.individualEid









firstName



individual.firstName

lastName


Yindividual.lastName

middleName



individual.middleName

typeCode



N/A



subTypeCode

HCPSubTypeCode

(TYP..*)



individual.typeCode

title

HCPTitle

(TIT.*)



individual.titleCode

prefix

HCPPrefix

(APP.*)



individual.prefixNameCode

suffix



N/A

gender

Gender

(.*)



individual.genderCode

specialties







typeCode

HCPSpecialty

(SP.W.*)



individual.speciality1


type


N/A


rank


get speciality with rank=1 


typeCode

HCPSpecialty

(SP.W.*)



individual.speciality2


type


N/A


rank


get speciality with rank=2 


typeCode

HCPSpecialty

(SP.W.*)



individual.speciality3


type


N/A


rank


get speciality with rank=3 

addresses







sourceAddressId


N/A


addressType


N/A


addressLine1


address.longLabel


addressLine2


address.longLabel2


addressLine3


N/A


stateProvince

AddressState

(DPT.W.*)



address.countyCode


city

Yaddress.city


zip


address.longPostalCode


country

Yaddress.country


rank


get address with rank=1 

identifiers







type


N/A


id


N/A

phones







type


N/A


number


individual.mobilePhone


rank


get phone with rank=1 

emails







type


N/A


email


individual.email


rank


get phone with rank=1 

contactAffiliationsno value in PFORCERX






type

RoleType

(TIH.W.*)



activity.role


primary


N/A


rank


get affiliation with rank=1 

contactAffiliations reference EntitycrosswalksONEKEY

workplace.workplaceEid

HCP & HCO













YentityTypeACTIVITY

For HCP full mapping check the HCP section above

Yvalidation.clientRequestIdHUB_GENERATED_ID

For HCO full mapping check the HCO section above

Yvalidation.processQ





Yvalidation.requestDate1970-01-01T00:00Z





Yvalidation.callDate1970-01-01T00:00Z
attributes



Yvalidation.requestProcessI

extDCRComment



validation.requestComment









country


YisoCod2

addresses







If the HCO address exists map to ONEKEY address

address (mapping HCO)


else






If the HCP address exists map to ONEKEY address

address (mapping HCP)

contactAffiliationsno value in PFORCERX






type

RoleType

(TIH.W.*)


activity.role


primary


N/A


rank


get affiliation with rank=1 


Triggers

Trigger action

Component

Action

Default time

REST callDCR Service: POST /dcrcreate DCRs in the ONEKEYAPI synchronous requests - realtime


Dependent components

Component

Usage

DCR Service 2Main component with flow implementation
Hub StoreDCR and Entities Cache 
" }, { "title": "OneKey: generate DCR Change Events (traceVR)", "pageID": "209950500", "pageLink": "/pages/viewpage.action?pageId=209950500", "content": "

Description

This process is triggered after the DCR was routed to Onekey based on the decision table configuration. The process of tracing the VR changes is based on the OneKey VR changes. During this process HUB, DCR Cache is triggered every <T> hour for SENT DCR's and check VR status using OneKey web service. After verification, the DCR Change event is generated. The DCR event is processed in the OneKey: process DCR Change Events and the DCR is updated in Reltio with Accepted or Rejected status.

Flow diagram

\"\"

Steps


Event Model

data class OneKeyDCREvent(val eventType: String? = null,
                          val eventTime: Long? = null,
                          val eventPublishingTime: Long? = null,
                          val countryCode: String? = null,
                          val dcrId: String? = null,
                          val targetChangeRequest: OneKeyChangeRequest,
)

data class OneKeyChangeRequest(
    val vrStatus : String? = null,
    val vrStatusDetail : String? = null,
    val oneKeyComment : String? = null,
    val individualEidValidated : String? = null,
    val workplaceEidValidated : String? = null,
    val vrTraceRequest : String? = null,
    val vrTraceResponse : String? = null,
)

Triggers

Trigger action

Component

Action

Default time

IN Timer (cron)dcr-service:TraceVRServicequery mongo to get all SENT DCR's related to the PFORCERX processevery <T> hour
OUT Eventsdcr-service:TraceVRServicegenerate the OneKeyDCREventevery <T> hour


Dependent components

Component

Usage

DCR ServiceMain component with flow implementation
Hub StoreDCR and Entities Cache 
" }, { "title": "OneKey: process DCR Change Events", "pageID": "209949303", "pageLink": "/display/GMDM/OneKey%3A+process+DCR+Change+Events", "content": "
\n
\n
\n
\n

Description

The process updates the DCR's based on the Change Request events received from [ONEKEY|VOD] (after trace VR method result). Based on the [IQVIA|VEEVA] Data Steward decision the state attribute contains relevant information to update DCR status. During this process also the comments created by IQVIA DS are retrieved and the relationship (optional step) between the DCR object and the newly created entity is created. DCR status is accepted only after the [ONEKEY|VOD] profile is created in Reltio, only then the Client will receive the ACCEPTED status. The process is checking Reltio with <T> delay and retries if the ETL load is still in progress waiting for [ONEKEY|VOD] profile. 

Flow diagram

\n
\n
\n
\n
\n
\n

OneKey variant

\"\"

\n
\n
\n
\n

Veeva variant: \"\"

\n
\n
\n
\n
\n
\n


Steps

  • OneKey: generate DCR Change Events (traceVR) publishes simple events to $env-internal-onekey-dcr-change-events-in: DCR_CHANGED
  • Events are aggregated in a time window (recommended the window length 24 hours) and the last event is returned to the process after the window is closed.
  • Events are processed in the Stream and based on the OneKeyDCREvent.OneKeyChangeRequest.vrStatus | VeevaDCREvent.VeevaChangeRequestDetails.vrStatus attribute decision is made
  • DCR is retrieved from the cache based on the _id of the DCR
  • If the event state is ACCEPTED
    • Get Reltio entity PfizerCustomerID by [ONEKEY|VOD] crosswalk
    • If such crosswalk entity exists in Reltio:
      • PfizerGlobalCustomerId is saved in Registry and will be returned to the Client 
      • During the process, the optional check is triggered - create the relation between the DCR object and newly created entities
        • if DCRRegistry contain an empty list of entityUris, or some of the newly created entity is not present in the list, the Relation between this object and the DCR has to be created
          • DCR entity is updated in Reltio and the relation between the processed entity and the DCR entity
            • Reltio source name (crosswalk. type): DCR
            • Reltio relation type: HCPtoDCR or HCOtoDCR (depending on the object type)
          • Newly created entities uris should be retrieved by the individualEidValidated or workplaceEidValidated (it may be both) attributes from the events that represent the HCP or HCO crosswalks.

      • The status in Reltio and in Mongo is updated

        DCR entity attributes

        Mapping for OneKey

        Mapping for Veeva

        VRStatusCLOSED
        VRStatusDetail

        state: ACCEPTED

        CommentsONEKEY comments ({VR.rsp.responseComments})
        ONEKEY ID = individualEidValidated or workplaceEidValidated        		VEEVA comments = VR.rsp.responseComments
        VEEVA ID = entityUris
        PfizerGlobalCustomerIdThis is required in ACCEPTED status

         

    • If the [ONEKEY|VOD] does not exist in Reltio
      • Regenerate the Event with a new timestamp to the input topic so this will be processed in the next <T> hours
      • Update the Reltio DCR status

        • DCR entity attributes

          Mapping

          VRStatusOPEN
          VRStatusDetail

          ACCEPTED

      • update the Mongo status to the OK_NOT_FOUND | VEEVA_NOT_FOUND and increase the "retryCounter" attribute
  • If the event state is REJECTED
    • If a Reltio DS has already seen this request, REJECT the DCR and end the flow (if the initial target type is Reltio)

      The status in Reltio and in Mongo is updated

      DCR entity attributes

      Mapping

      VRStatusCLOSED
      VRStatusDetail

      state: REJECTED

      Comments[ONEKEY|VOD] comments ({VR.rsp.responseComments})
    • If this is based on the routing table and it was never sent to the Reltio DS, then create the DCR workflow and send this to the Reltio DS. Add the information comment that this was Rejected by the OneKey, so now Reltio DS has to decide if this should be REJECTED or APPLIED in Reltio. Add the comment that this is not possible to execute the sendTo3PartyValidation button in this case. Steps:
      • Check if the initial target type is [ONEKEY|VOD]
      • Use the DCR Request that was initially received from PforceRx and is a Domain Model request (after validation) 
      • Send the DCR to Reltio the service returns the following response:
        • ACCEPTED (change request accepted by Reltio)
          • update the status to DS_ACTION_REQUIERED and in the comment add the following: "This DCR was REJECTED by the [ONEKEY|VOD] Data Steward with the following comment: <[ONEKEY|VOD] reject comment>. Please review this DCR in Reltio and APPLY or REJECT. It is not possible to execute the sendTo3PartyValidation button in this case"
          • initialize new Workflow in Reltio with the comment.
          • save data in the DCR entity status in Reltio and update Mongo DCR Registry with workflow ID and other attributes that were used in this Flow.
        • REJECTED  (failure or error response from Reltio)
          • CLOSE the DCR with the information that DCR was REJECTED by the [ONEKEY|VOD] and Reltio also REJECTED the DCR. Add the error message from both systems in the comment. 

Triggers

Trigger action

Component

Action

Default time

IN Events incoming 

dcr-service-2:DCROneKeyResponseStream

dcr-service-2:DCRVeevaResponseStream ($env-internal-veeva-dcr-change-events-in)

process publisher full change request events in the streamrealtime: events stream processing 

Dependent components

Component

Usage

DCR Service 2Main component with flow implementation
ManagerReltio Adapter  - API operations
PublisherEvents publisher generates incoming events
Hub StoreDCR and Entities Cache 
\n
\n
\n
" }, { "title": "Reltio: create DCR method - direct", "pageID": "209949292", "pageLink": "/display/GMDM/Reltio%3A+create+DCR+method+-+direct", "content": "

Description

Rest API method exposed in the Manager component responsible for submitting the Change Request to Reltio

Flow diagram

\"\"

Steps

  1. Receive the DCR request generated by DCR Service 2 component
  2. Depending on the Action execute the method in the Manager component:
    1. insert - Execute standard Create/Update HCP/HCO/MCO operation with additional changeRequest.id parameter
    2. update - Execute Update Attributes operation with additional changeRequest.id parameter
      1. the combination of IGNORE_ATTRIBUTE & INSERT_ATTRIBUTE once updating existing parameter in Reltio
      2. the INSERT_ATTRIBUTE once adding new attribute to Reltio
    3. delete - Execute Update Attribute operation with additional changeRequest.id parameter
      1. the UPDATE_END_DATE on the entity to inactivate this profile
  3. Based on the Reltio response the DCR Response is returned:
    1. REQUEST_ACCEPTED - Reltio processed the request successfully 
    2. REQUEST_FAILED - Reltio returned the exception, Client will receive the detailed description in the errorMessage

Triggers

Trigger action

Component

Action

Default time

REST callDCR Service: POST /dcr2Create change Requests in ReltioAPI synchronous requests - realtime


Dependent components

Component

Usage

DCR ServiceMain component with flow implementation
Hub StoreDCR and Entities Cache 
" }, { "title": "Reltio: process DCR Change Events", "pageID": "209949300", "pageLink": "/display/GMDM/Reltio%3A+process+DCR+Change+Events", "content": "

Description

The process updates the DCR's based on the Change Request events received from Reltio(publishing). Based on the Data Steward decision the state attribute contains relevant information to update DCR status. During this process also the comments created by DS are retrieved and the relationship (optional step) between the DCR object and the newly created entity is created.


Flow diagram

\"\"

Steps

Triggers

Trigger action

Component

Action

Default time

IN Events incoming dcr-service-2:DCRReltioResponseStreamprocess publisher full change request events in the streamrealtime: events stream processing 

Dependent components

Component

Usage

DCR Service

DCR Service 2

Main component with flow implementation
ManagerReltio Adapter  - API operations
PublisherEvents publisher generates incoming events
Hub StoreDCR and Entities Cache 
" }, { "title": "Reltio: Profiles created by DCR", "pageID": "510266969", "pageLink": "/display/GMDM/Reltio%3A+Profiles+created+by+DCR", "content": "
DCR typeApproval/Reject Record visibility in MDMCrosswalk TypeCrosswalk ValueSource
DCR create for HCP/HCOApproved by OneKey/VODHCP/HCO created in MDMONEKEY|VODonekey id ONEKEY|VOD
Approved by DSRHCP/HCO created in MDMSystem source name from DCR (KOL_OneView, PforceRx, etc)DCR IDSystem source name from DCR (KOL_OneView, PforceRx, etc)
DCR edit for HCP/HCOApproved by OneKey/VODHCP/HCO requested attribute updated in MDMONEKEY|VOD
ONEKEY|VOD
Approved by DSRHCP/HCO requested attribute updated in MDMReltioentity uriReltio
DCR edit for HCPaddress/HCO addressApproved by OneKey/VODNew address created in MDM, existing address marked as inactiveONEKEY|VOD
ONEKEY|VOD
Approved by DSRNew address created in MDM, existing address marked as inactiveReltioentity uriReltio
" }, { "title": "Veeva DCR flows", "pageID": "379332475", "pageLink": "/display/GMDM/Veeva+DCR+flows", "content": "

Description

The process is responsible for creating DCRs which are stored (Store VR) to be further transferred and processed by Veeva. Changes can be suggested by the DS using "Suggest" operation in Reltio and "Send to Third Party Validation" button. All DCRs are saved in the dedicated collection in HUB Mongo DB, required to gather metadata and trace the changes for each DCR request. During this process, the communication to Veeva Opendata is established via S3/SFTP communication. SubmitVR operation is executed to create a new ZIP files with DCR requests spread across multiple CSV files. The TraceVR operation is executed to check if Veeva responded to initial DCR Requests via ZIP file placed Inbound S3 dir. 

The process is divided into 3 sections:

  1. Create DCR request - Veeva
  2. Submit DCR Request - Veeva
  3. Trace Validation Request - Veeva

The below diagram presents an overview of the entire process. Detailed descriptions are available in the separated subpages.

Business process diagram for R1 phase

\"\"


Flow diagram

\"\"

Steps

Triggers

DCR service 2 is being triggered via /dcr API calls which are triggered by Data Stewards actions (R1 phase) → "Suggests 3rd party validation" which pushes DCR from Reltio to HUB.

Dependent components

Described in the separated sub-pages for each process.

Design document for HUB development 

  1. Design → VeevaOpenData-implementation.docx

  2. Reltio HUB-VOD mapping → VeevaOpenDataAPACDataDictionary.xlsx
  3. VOD model description (v4) → Veeva_OpenData_APAC_Data_Dictionary v4.xlsx
" }, { "title": "Create DCR request - Veeva", "pageID": "386814533", "pageLink": "/display/GMDM/Create+DCR+request+-+Veeva", "content": "

Description

The process of creating new DCR requests to the Veeva OpenData. During this process, new DCRs are created in DCRregistryVeeva mongo collection.

Flow diagram

\"\"

Steps

Mappings

DCR domain model→ VOD mapping file: VeevaOpenDataAPACDataDictionary-mmor-mapping.xlsx

Veeva integration guide

\"\"

" }, { "title": "Submit DCR Request - Veeva", "pageID": "379333348", "pageLink": "/display/GMDM/Submit+DCR+Request+-+Veeva", "content": "

Description

The process of submitting new validation requests to the Veeva OpenData service via VeevaAdapter (communication with S3/SFTP) based on DCRRegistryVeeva mongo collection . During this process, new DCRs are created in VOD system.

Flow diagram


\"\"

Steps

Veeva DCR service flow:

SFTP integration service flow:

Triggers

Trigger actionComponentActionDefault time
Spring scheduler
mdm-veeva-dcr-service:VeevaDCRRequestSenderprepare ZIP files for VOD systemCalled every specified interval

Dependent components

ComponentUsage
Veeva adapterUpload DCR request to s3 location
" }, { "title": "Trace Validation Request - Veeva", "pageID": "379333358", "pageLink": "/display/GMDM/Trace+Validation+Request+-+Veeva", "content": "

Description

The process of tracing the VR changes based on the Veeva VR changes. During this process HUB, DCRRegistryVeeva Cache is triggered every <T> hour for SENT DCR's and check VR status using Veeva Adapter (s3/SFTP integration). After verification DCR event is sent to DCR Service 2  Veeva response stream.

Flow diagram


\"\"


Steps


Triggers

Trigger actionComponentActionDefault time
IN Spring schedulermdm-veeva-dcr-service:VeevaDCRRequestTracestart trace validation request processevery <T> hour
OUT Kafka topicmdm-dcr-service-2:VeevaResponseStreamupdate DCR status in Reltio, create relationsinvokes Kafka producer for each veeva DCR response

Dependent components

ComponentUsage
DCR Service 2Process response event
" }, { "title": "Veeva: create DCR method (storeVR)", "pageID": "379332642", "pageLink": "/pages/viewpage.action?pageId=379332642", "content": "

Description

Rest API method exposed in the Veeva DCR Service component responsible for creating new DCR requests specific to Veeva OpenData (VOD) and storing them in dedicated collection for further submit. Since VOD enables communication only via S3/SFTP, it's required to use dedicated mechanism to actually trigger CSV/ZIP file creation and file placement in outbound directory. This will periodic call to Submit VR method will be scheduled once a day (with cron) which will in the end call VeevaAdapter with method createChangeRequest.

Flow diagram

\"\"

Steps


  1. Receive the API request
  2. Validate initial request
    1. check if the Veeva crosswalk exists once there is an update on the profile
    2. otherwise it's required to prepare DCR to create new Veeva profile
    3. If there is any formal attribute missing or incorrect: skip request
  3. Then the DCR is mapped to Veeva Request by invoking mapper between HUB DCR → VEEVA model 
    1. For mapping purpose below mapping table should be used 
    2. If there is not proper LOV mapping between HUB and Veeva, default fallback should be set to question mark → ?  
  4. Once proper request has been created, it should be stored as a VeevaVRDetails entry in dedicated DCRRegistryVeeva collection to be ready for actually send via Submit VR job and for future tracing purposes
  5. Prepare return response for initial API request with below logic
    1. Generate sample request after successful mongo insert →  generateResponse(dcrRequest, RequestStatus.REQUEST_ACCEPTED, null, null)
    2. Generate error when validation or exception →  generateResponse(dcrRequest, RequestStatus.REQUEST_FAILED, getErrorDetails(), null);

Mapping HUB DCR → Veeva model 

ReltioHUBVEEVA
Attribute PathDetailsDCR Request pathDetailsFile NameField NameRequired for Add Request?Required for Change Request?DescriptionReference (RDM/LOV)NOTE
HCO
N/A
Mongo Generated ID for this DCR | Kafka KEYonce mapping from HUB Domain DCRRequest take this from DCRRequestD.dcrRequestId: String, // HUB DCR request id - Mongo ID - required in ONEKEY servicechange_requestdcr_keyYYCustomer's internal identifier for this request

Change Requests comments 
extDCRComment
change_requestdescriptionYYRequester free-text comments explaining the DCR

targetChangeRequest.createdBy
createdBy
change_requestcreated_byYYFor requestor identification

N/A
if new objects - ADD, if veeva ID CHANGE
change_requestchange_request_typeYYADD_REQUEST or CHANGE_REQUEST

N/Adepends on suggested changes (check use-cases)main entity object type HCP or HCO
change_requestentity_typeYNHCP or HCOEntityType
N/A
Mongo Generated ID for this DCR | Kafka KEY
change_request_hcodcr_keyYYCustomer's internal identifier for this request

Reltio Uri and Reltio Typewhen insert new profileentities.HCO.updateCrosswalk.type (Reltio)
entities.HCO.updateCrosswalk.value (Reltio id)
and
refId.entityURI		concatenate Reltio:rvu44dmchange_request_hcoentity_keyYYCustomer's internal HCO identifier

Crosswalks - VEEVA crosswalkwhen update on VEEVAentities.HCO.updateCrosswalk.type (VEEVA)
entities.HCO.updateCrosswalk.value (VEEVA ID)
change_request_hcovid__vYNVeeva ID of existing HCO to update; if blank, the request will be interpreted as an add request

configuration/entityTypes/HCO/attributes/OtherNames/attributes/Namefirst elementTODO - add new attribute
change_request_hcoalternate_name_1__vYN


??
??
change_request_hcobusiness_type__vYN
HCOBusinessTypeTO BE CONFIRMED
configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/FacilityType
HCO.subTypeCode
change_request_hcpmajor_class_of_trade__vNN
COTFacilityType

In PforceRx - Account Type, more info: \n MR-9512\n -\n Getting issue details...\n STATUS\n

configuration/entityTypes/HCO/attributes/Name
name
change_request_hcocorporate_name__vNY


configuration/entityTypes/HCO/attributes/TotalLicenseBeds
TODO - add new attribute
change_request_hcocount_beds__vNY


configuration/entityTypes/HCO/attributes/Email/attributes/Emailemail with rank 1emails
change_request_hcoemail_1__vNN


configuration/entityTypes/HCO/attributes/Email/attributes/Emailemail with rank 2
change_request_hcoemail_2__vNN


configuration/entityTypes/HCO/attributes/Phone/attributes/Numberphone type TEL.FAX with best rankphones
change_request_hcofax_1__vNN


configuration/entityTypes/HCO/attributes/Phone/attributes/Numberphone type TEL.FAX with worst rank
change_request_hcofax_2__vNN


configuration/entityTypes/HCO/attributes/StatusDetail
TODO - add new attribute
change_request_hcohco_status__vNN
HCOStatus
configuration/entityTypes/HCO/attributes/TypeCode
typecode
change_request_hcohco_type__vNN
HCOType
configuration/entityTypes/HCO/attributes/Phone/attributes/Numberphone type TEL.OFFICE with best rankphones
change_request_hcophone_1__vNN


configuration/entityTypes/HCO/attributes/Phone/attributes/Numberphone type TEL.OFFICE with worst rank
change_request_hcophone_2__vNN


configuration/entityTypes/HCO/attributes/Phone/attributes/Numberphone type TEL.OFFICE with worst rank
change_request_hcophone_3__vNN


configuration/entityTypes/HCO/attributes/Country
DCRRequest.country
change_request_hcoprimary_country__vNN


configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialtyelements from COT specialties
change_request_hcospecialty_1__vNN


configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty
change_request_hcospecialty_10__vNN
Speciality
configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty
change_request_hcospecialty_2__vNN

configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty
change_request_hcospecialty_3__vNN

configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty
change_request_hcospecialty_4__vNN

configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty
change_request_hcospecialty_5__vNN

configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty
change_request_hcospecialty_6__vNN

configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty
change_request_hcospecialty_7__vNN

configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty
change_request_hcospecialty_8__vNN

configuration/entityTypes/HCO/attributes/ClassofTradeN/attributes/Specialty
change_request_hcospecialty_9__vNN

configuration/entityTypes/HCO/attributes/Website/attributes/WebsiteURLfirst elementwebsiteURL
change_request_hcoURL_1__vNN


configuration/entityTypes/HCO/attributes/Website/attributes/WebsiteURLN/AN/A
change_request_hcoURL_2__vNN


HCP
 
N/A
Mongo Generated ID for this DCR | Kafka KEY
change_request_hcpdcr_keyYYCustomer's internal identifier for this request

Reltio Uri and Reltio Typewhen insert new profileentities.HCO.updateCrosswalk.type (Reltio)
entities.HCO.updateCrosswalk.value (Reltio id)
and
refId.entityURI		concatenate Reltio:rvu44dmchange_request_hcpentity_keyYYCustomer's internal HCP identifier

configuration/entityTypes/HCP/attributes/Country
DCRRequest.country
change_request_hcpprimary_country__vYY


Crosswalks - VEEVA crosswalkwhen update on VEEVAentities.HCO.updateCrosswalk.type (VEEVA)
entities.HCO.updateCrosswalk.value (VEEVA ID)
change_request_hcpvid__vNY


configuration/entityTypes/HCP/attributes/FirstName
firstName
change_request_hcpfirst_name__vYN


configuration/entityTypes/HCP/attributes/Middle
middleName
change_request_hcpmiddle_name__vNN


configuration/entityTypes/HCP/attributes/LastName
lastName
change_request_hcplast_name__vYN


configuration/entityTypes/HCP/attributes/Nickname
TODO - add new attribute
change_request_hcpnickname__vNN


configuration/entityTypes/HCP/attributes/Prefix
prefix
change_request_hcpprefix__vNN
HCPPrefix
configuration/entityTypes/HCP/attributes/SuffixName
suffix
change_request_hcpsuffix__vNN


configuration/entityTypes/HCP/attributes/Title
title
change_request_hcpprofessional_title__vNN
HCPProfessionalTitle
configuration/entityTypes/HCP/attributes/SubTypeCode
subTypeCode
change_request_hcphcp_type__vYN
HCPType
configuration/entityTypes/HCP/attributes/StatusDetail
TODO - add new attribute
change_request_hcphcp_status__vNN
HCPStatus
configuration/entityTypes/HCP/attributes/AlternateName/attributes/FirstName
TODO - add new attribute
change_request_hcpalternate_first_name__vNN


configuration/entityTypes/HCP/attributes/AlternateName/attributes/LastName
TODO - add new attribute
change_request_hcpalternate_last_name__vNN


configuration/entityTypes/HCP/attributes/AlternateName/attributes/MiddleName
TODO - add new attribute
change_request_hcpalternate_middle_name__vNN


??
TODO - add new attribute
change_request_hcpfamily_full_name__vNN

TO BE CONFRIMED
configuration/entityTypes/HCP/attributes/DoB
birthYear
change_request_hcpbirth_year__vNN


configuration/entityTypes/HCP/attributes/Credential/attributes/Credentialby rank 1TODO - add new attribute
change_request_hcpcredentials_1__vNN

TO BE CONFIRMED
configuration/entityTypes/HCP/attributes/Credential/attributes/Credential2TODO - add new attribute
change_request_hcpcredentials_2__vNN

In reltio there is attribute but not used
configuration/entityTypes/HCP/attributes/Credential/attributes/Credential3TODO - add new attribute
change_request_hcpcredentials_3__vNN

                            "uri": "configuration/entityTypes/HCP/attributes/Credential/attributes/Credential",
configuration/entityTypes/HCP/attributes/Credential/attributes/Credential4TODO - add new attribute
change_request_hcpcredentials_4__vNN

                            "lookupCode": "rdm/lookupTypes/Credential",
configuration/entityTypes/HCP/attributes/Credential/attributes/Credential5TODO - add new attribute
change_request_hcpcredentials_5__vNN
HCPCredentials                            "skipInDataAccess": false
??
TODO - add new attribute
change_request_hcpfellow__vNN
BooleanReferenceTO BE CONFRIMED
configuration/entityTypes/HCP/attributes/Gender
gender
change_request_hcpgender__vNN
HCPGender
?? Education ??
TODO - add new attribute
change_request_hcpeducation_level__vNN
HCPEducationLevelTO BE CONFRIMED
configuration/entityTypes/HCP/attributes/Education/attributes/SchoolName
TODO - add new attribute
change_request_hcpgrad_school__vNN


configuration/entityTypes/HCP/attributes/Education/attributes/YearOfGraduationTODO - add new attribute
change_request_hcpgrad_year__vNN


??


change_request_hcphcp_focus_area_10__vNN

TO BE CONFRIMED
??


change_request_hcphcp_focus_area_1__vNN


??


change_request_hcphcp_focus_area_2__vNN


??


change_request_hcphcp_focus_area_3__vNN


??


change_request_hcphcp_focus_area_4__vNN


??


change_request_hcphcp_focus_area_5__vNN


??


change_request_hcphcp_focus_area_6__vNN


??


change_request_hcphcp_focus_area_7__vNN


??


change_request_hcphcp_focus_area_8__vNN


??


change_request_hcphcp_focus_area_9__vNN
HCPFocusArea
??


change_request_hcpmedical_degree_1__vNN

TO BE CONFRIMED
??


change_request_hcpmedical_degree_2__vNN
HCPMedicalDegree
configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialtyby rank from 1 to 100specialties
change_request_hcpspecialty_1__vYN


configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialtyspecialties
change_request_hcpspecialty_10__vNN


configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialtyspecialties
change_request_hcpspecialty_2__vNN


configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialtyspecialties
change_request_hcpspecialty_3__vNN


configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialtyspecialties
change_request_hcpspecialty_4__vNN


configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialtyspecialties
change_request_hcpspecialty_5__vNN


configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialtyspecialties
change_request_hcpspecialty_6__vNN


configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialtyspecialties
change_request_hcpspecialty_7__vNN


configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialtyspecialties
change_request_hcpspecialty_8__vNN


configuration/entityTypes/HCP/attributes/Specialities/attributes/Specialtyspecialties
change_request_hcpspecialty_9__vNN
Specialty
configuration/entityTypes/HCP/attributes/WebsiteURL
TODO - add new attribute
change_request_hcpURL_1__vNN


ADDRESS


Mongo Generated ID for this DCR | Kafka KEY
change_request_addressdcr_keyYYCustomer's internal identifier for this request

Reltio Uri and Reltio Typewhen insert new profileentities.HCP OR HCO.updateCrosswalk.type (Reltio)
entities.HCP OR HCO.updateCrosswalk.value (Reltio id)
and
refId.entityURI		concatenate Reltio:rvu44dmchange_request_addressentity_keyYYCustomer's internal HCO/HCP identifier

attributes/Addresses/attributes/PfizerAddressID
address.refId
change_request_addressaddress_keyYYCustomer's internal address identifier

attributes/Addresses/attributes/AddressLine1
addressLine1
change_request_addressaddress_line_1__vYN


attributes/Addresses/attributes/AddressLine2
addressLine2
change_request_addressaddress_line_2__vNN


attributes/Addresses/attributes/AddressLine3
addressLine3
change_request_addressaddress_line_3__vNN


N/A
N/AAchange_request_addressaddress_status__vNN
AddressStatus
attributes/Addresses/attributes/AddressType
addressType
change_request_addressaddress_type__vYN
AddressType
attributes/Addresses/attributes/StateProvince
stateProvince
change_request_addressadministrative_area__vYN
AddressAdminArea
attributes/Addresses/attributes/Country
country
change_request_addresscountry__vYN


attributes/Addresses/attributes/City
city
change_request_addresslocality__vYY


attributes/Addresses/attributes/Zip5
zip
change_request_addresspostal_code__vYN


attributes/Addresses/attributes/Source/attributes/SourceName
attributes/Addresses/attributes/Source/attributes/SourceAddressID		when VEEVA map VEEVA ID to sourceAddressId
change_request_addressvid__vNY


map from
relationTypes/OtherHCOtoHCOAffiliations
or
relationTypes/ContactAffiliations
This will be HCP.ContactAffiliation or HCO.OtherHcoToHCO affiliation








Mongo Generated ID for this DCR | Kafka KEY
change_request_parenthcodcr_keyYYCustomer's internal identifier for this request



HCO.otherHCOAffiliations.relationUri
or
HCP.contactAffiliations.relationUri		 (from Domain model)
information about Reltio Relation ID		change_request_parenthcoparenthco_keyYYCustomer's internal identifier for this relationshipRELATION ID


KEY entity_key from HCP or HCO (start object)
change_request_parenthcochild_entity_keyYYChild Identifier in the HCO/HCP fileSTART OBJECT ID
endObject entity uri mapped to refId.EntityURITargetObjectId
KEY entity_key from HCP or HCO (end object, by affiliation)
change_request_parenthcoparent_entity_keyYYParent identifier in the HCO fileEND OBJECT ID

changes in Domain model mappingmap Reltion.Source.SourceName - VEEVA
map Relation.Source.SourceValue - VEEVA ID		add to Domain model
map if relation is from VEEVA ID 		change_request_parenthcovid__vNY




start object entity type 
change_request_parenthcoentity_type__vYN


attributes/RelationType/attributes/PrimaryAffiliation
if is primary
TODO - add new attribute to otherHcoToHCO
change_request_parenthcois_primary_relationship__vNN
BooleanReference


HCO_HCO or HCP_HCO
change_request_parenthcohierarchy_type__v


RelationHierarchyType
attributes/RelationType/attributes/RelationshipDescription
type from affiliation
based on ContactAffliation or OtherHCOToHCO affiliation		I think it will be 14-Emploted for HCP_HCO
and 4-Manages for HCO_HCO
but maybe we can map from affiliation.type		change_request_parenthcorelationship_type__vYN
RelationType


Mongo collection

All DCRs initiated by the dcr-service-2 API and to be sent to Veeva will be stored in Mongo in new collection DCRRegistryVeeva. The idea is to gather all DCRs requested by the client through the day and schedule ‘SubmitVR’ process that will communicate with Veeva adapter.

Typical use case: 


In this store we are going to keep both types of DCRs:

\n
initiated by PforceRX - PFORCERX_DCR("PforceRxDCR")\ninitiated by Reltio SubmitVR - SENDTO3PART_DCR("ReltioSuggestedAndSendTo3PartyDCR");
\n


Store class idea:


VeevaVRDetails
\n
@Document("DCRRegistryVEEVA")\n@JsonIgnoreProperties(ignoreUnknown = true)\n@JsonInclude(JsonInclude.Include.NON_NULL)\ndata class VeevaVRDetails(\n    @JsonProperty("_id")\n    @Id\n    val id: String? = null,\n    val type: DCRType,\n    val status: DCRRequestStatusDetails,\n    val createdBy: String? = null,\n    val createTime: ZonedDateTime? = null,\n    val endTime: ZonedDateTime? = null,\n    val veevaRequestTime: ZonedDateTime? = null,\n    val veevaResponseTime: ZonedDateTime? = null,\n    val veevaRequestFileName: String? = null\n    val veevaResponseFileName: String? = null    val veevaResponseFileTime: ZonedDateTime? = null\n    val country: String? = null,\n    val source: String? = null,\n    val extDCRComment: String? = null, // external DCR Comment (client comment)\n    val trackingDetails: List<DCRTrackingDetails> = mutableListOf(),\n\n    RAW FILE LINES mapped from DCRRequestD to Veeva model\n    val veevaRequest:\n            val change_request_csv: String,\n            val change_request_hcp_csv: String\n            val change_request_hco_csv: List<String>\n            val change_request_address_csv: List<String>\n            val change_request_parenthco_csv: List<String>\n\n    RAW FILE LINES mapped from Veeva Response model\n    val veevaResponse:\n            val change_request_response_csv: String,\n            val change_request_response_hcp_csv: String\n            val change_request_response_hco_csv: List<String>\n            val change_request_response_address_csv: List<String>\n            val change_request_response_parenthco_csv: List<String>\n)
\n

Mapping Reltio canonical codes → Veeva source codes

There are a couple of steps performed to find out a mapping for canonical code from Reltio to source code understood by VOD. Below steps are performed (in this order) once a code is found. 

Veeva Defaults 

Configuration is stored in mdm-config-registry > config-hub/stage_apac/mdm-veeva-dcr-service/defaults

The purpose of these logic is to select one of possible multiple source codes on VOD end for a single code on Pfizer side (1:N). The other scenario is when there is no actual source code for a canonical code on VOD end (1:0), however this is usually covered by fallback code logic.

There are a couple of files, each containing source codes for a specific attribute. The ones related to HCO.Specialty and HCP.Specialty have logic which selects proper code.

RDM lookups with RegExp

The main logic which is used to find out proper source code for canonical code. We're using codes configured in RDM, however mongo collection LookupValues are used. For specific canonical code (code) we looking for sourceMappings with source = VOD. Often country is embedded within source code so we're applying regexpConfig (more in Veeva Fallback section) to extract specific source code for particular country.

Veeva Fallback

Configuration is stored in mdm-config-registry > config-hub/stage_apac/mdm-veeva-dcr-service/fallback


Triggers

Trigger action

Component

Action

Default time

REST callmdm-veeva-dcr-service: POST /dcr → veevaDCRService.createChangeRequest(request)

Creates DCR and stores it in collection without actual send to Veeva. 

API synchronous requests - realtime


Dependent components

Component

Usage

DCR Service 2Main component with flow implementation
Hub StoreDCR and Entities Cache 
" }, { "title": "Veeva: create DCR method (submitVR)", "pageID": "386796763", "pageLink": "/pages/viewpage.action?pageId=386796763", "content": "

Description

Gather all stored DCR entities in DCRRegistryVeeva collection (status = NEW) and sends them via S3/SFTP to Veeva OpenData (VOD). This method triggers CSV/ZIP file creation and file placement in outbound directory. This method is triggered from cron which invokes VeevaDCRRequestSender.sendDCRs() from the Veeva DCR Service 

Flow diagram

\"\"

Steps


  1. Receive the API request via scheduled trigger, usually every 24h (senderConfiguration.schedulerConfig.fixedDelay) at specific time of day (senderConfiguration.schedulerConfig.initDelay)
  2. All DCR entities (VeevaVRDetails) with status NEW are being retrieved from DCRRegistryVeeva collection 
  3. Then VeevaCreateChangeRequest object is created which aggregates all CSV content which should be placed in actual CSV files. 
    1. Each object contains only DCRs specific for country
    2. Each country has its own S3/SFTP directory structure as well as dedicated SFTP server instance
  4. Once CSV files are created with header and content, they are packed into single ZIP file
  5. Finally ZIP file is placed in outbound S3 directory
  6. If file was placed
    1. successfuly - then VeevaChangeRequestACK status = SUCCESS
    2. otherwise - then VeevaChangeRequestACK status = FAILURE and process ends
  7. Finally, status of VeevaVRDetails entity in DCRRegistryVeeva collection is updated and set to SENT_TO_VEEVA

Triggers

Trigger action

Component

Action

Default time

Timer (cron)mdm-veeva-dcr-service: VeevaDCRRequestSender.sendDCRs()

Takes all unsent entities (status = NEW) from Veeva collection and actually puts file on S3/SFTP directory via veevaAdapter.createDCRs



Usually every 24h (senderConfiguration.schedulerConfig.fixedDelay) at specific time of day (senderConfiguration.schedulerConfig.initDelay)


Dependent components

Component

Usage

DCR Service 2Main component with flow implementation
Hub StoreDCR and Entities Cache 
" }, { "title": "Veeva: generate DCR Change Events (traceVR)", "pageID": "379329922", "pageLink": "/pages/viewpage.action?pageId=379329922", "content": "

Description

The process is responsible for gathering DCR responses from Veeva OpenData (VOD). Responses are provided via CSV/ZIP files placed on S3/SFTP server in inbound directory which are specific for each country. During this process files should be retrieved, mapped from VOD to HUB DCR model and published to Kafka topic to be properly processed by DCR Service 2, Veeva: process DCR Change Events.

Flow diagram

\"\"

Source: Lucid

Steps

  1. Method is trigger via cron, usually every 24h (traceConfiguration.schedulerConfig.fixedDelay) at specific time of day (traceConfiguration.schedulerConfig.initDelay)
  2. For each country, each inbound directory in scanned for ZIP files
  3. Each ZIP files (<country>_DCR_Response_<Date>.zip) should be unpacked and processed. A bunch of CSV files should be extracted. Specifically:
    1. change_request_response.csv → it's a manifest file with general information in specific columns
      1. dcr_key → ID of DCR which was established during DCR request creation 
      2. entity_key → ID of entity in Reltio, the same one we provided during DCR request creation
      3. entity_type → type of entity (HCO, HCP) which is being modified via this DCR
      4. resolution → has information whether DCR was accepted or rejected. Full list of values is below.

        1. resolution value

          Description

          CHANGE_PENDING

          This change is still processing and hasn't been resolved

          CHANGE_ACCEPTED

          This change has been accepted without modification

          CHANGE_PARTIAL

          This change has been accepted with additional changes made by the steward, or some parts of the change request have been rejected

          CHANGE_REJECTED

          This change has been rejected in its entirety

          CHANGE_CANCELLED

          This change has been cancelled

      5. change_request_type 
        1. change_request_type valueDescription
          ADD_REQUEST

          whether DCR caused to create new profile in VOD with new vid__v  (Veeva id)

          CHANGE_REQUEST

          just update of existing profile in VOD with existing and already known vid__v (Veeva id)

    2. change_request_hcp_response.csv - contains information about DCR related to HCP
    3. change_request_hco_response.csv - contains information about DCR related to HCO
    4. change_request_address_response.csv - contains information about DCR related to addresses which are related to specific HCP or HCO
    5. change_request_parenthco_response.csv - contains information about DCR which correspond to relations between HCP and HCO, and HCO and HCO
    6. File with log: <country>_DCR_Request_Job_Log.csv can be skipped. It does not contain any useful information to be processed automatically
  4. For all DCR responses from VOD, we need to get corresponding DCR entity (VeevaVRDetails)from collection DCRRegistryVeeva should be selected. 
  5. In general, specific response files are not that important (VOD profiles updates will be ingested to HUB via ETL channel) however when new profiles are created (change_request_response.csv.change_request_type = ADD_REQUEST) we need to extract theirs Veeva ID. 
    1. We need to deep dive into change_request_hcp_response.csv or change_request_hco_response.csv to find vid__v (Veeva ID) for specific dcr_key 
    2. This new Veeva ID should be stored in VeevaDCREvent.vrDetails.veevaHCPIds

    3. It should be further used as a crosswalk value in Reltio:

      1. entities.HCO.updateCrosswalk.type (VEEVA)
      2. entities.HCO.updateCrosswalk.value (VEEVA ID)
  6. Once data has been properly mapped from Veeva to HUB DCR model, new VeevaDCREvent entity should be created and published to dedicated Kafka topic $env-internal-veeva-dcr-change-events-in
    1. Please be advised, when the status of resolution is not final (CHANGE_ACCEPTED, CHANGE_REJECTED, CHANGE_CANCELLED, CHANGE_PARTIAL) we should not sent event to DCR-service-2
  7. Then for each successfully processed DCR entity (VeevaVRDetails) in Mongo  DCRRegistryVeeva collection should be updated 

    1. Veeva CSV: resolution

      Mongo: DCRRegistryVeeva 

      Entity: VeevaVRDetails.status: DCRRequestStatusDetails

      Topic: $env-internal-veeva-dcr-change-events-in

      Event: VeevaDCREvent.vrDetails.vrStatus

      Topic: $env-internal-veeva-dcr-change-events-in

      Event: VeevaDCREvent.vrDetails.vrStatusDetail

      CHANGE_PENDING

      status should not be updated at all (stays as SENT)

      		do not send events to DCR-service-2 do not send events to DCR-service-2 

      CHANGE_ACCEPTED

      		ACCEPTEDCLOSEDACCEPTED

      CHANGE_PARTIAL

      ACCEPTED

      		CLOSED

      ACCEPTED

      resolutionNotes / veevaComment should contain more information what was rejected by VEEVA DS

      CHANGE_REJECTED

      		REJECTEDCLOSEDREJECTED

      CHANGE_CANCELLED

      		REJECTEDCLOSEDREJECTED
  8. Once files are processed, ZIP file should be moved from inbound to archive directory


Event VeevaDCREvent Model

\n
data class VeevaDCREvent (val eventType: String? = null,\n                          val eventTime: Long? = null,\n                          val eventPublishingTime: Long? = null,\n                          val countryCode: String? = null,\n                          val dcrId: String? = null,\n                          val vrDetails: VeevaChangeRequestDetails)\n\ndata class VeevaChangeRequestDetails (\n    val vrStatus: String? = null, - HUB CODEs\n    val vrStatusDetail: String? = null, - HUB CODEs\n    val veevaComment: String? = null,\n    val veevaHCPIds: List<String>? = null,\n    val veevaHCOIds: List<String>? = null)
\n


Triggers

Trigger action

Component

Action

Default time

IN Timer (cron)mdm-veeva-dcr-service: VeevaDCRRequestTrace.traceDCRs()get DCR responses from S3/SFTP directory, extract CSV files from ZIP file and publish events to kafka topic

every <T> hour

usually every 6h (traceConfiguration.schedulerConfig.fixedDelay) at specific time of day (traceConfiguration.schedulerConfig.initDelay)

OUT Events on Kafka Topic

mdm-veeva-dcr-service: VeevaDCRRequestTrace.traceDCRs()

$env-internal-veeva-dcr-change-events-in

VeevaDCREvent event published to topic to be consumed by DCR Service 2

every <T> hour

usually every 6h (traceConfiguration.schedulerConfig.fixedDelay) at specific time of day (traceConfiguration.schedulerConfig.initDelay)


Dependent components

Component

Usage

DCR Service 2Main component with flow implementation
Hub StoreDCR and Entities Cache 
" } ]