✏️ Cloning

Reasons for Cloning

DevOps

Cloning is part of the AIForged DevOps Lifecycle. It assists AIForged developers to clone a “Live” project / service to at “Dev” project / service. Then the developer can make changes to settings and training data and test the impact the change will have on the “Live” project.

It can also assist with making “demo” copies of a project.

Clone can only be done on the Production environment. The cloning scheduler is not enabled on Sandbox or Development environments.

Demos or Other

Cloning can be done across tenants and projects to demo or research proposes.

Cloining Scheduler

Cloning run as a scheduled backend job. Once cloning is configured the scheduler will run cloning. Schedules can be configured for daily or weekly runs. If you require ad-hoc cloning, we suggested to schedule it for daily and then disable the schedule once it started.

How to create and change the schedule

You need to create cloning options first. Once that is done the cloning schedule will be created and appear in the service settings / definitions view.

How to change the options and validation checks

This will create cloning options and initiate cloning schedule.

Cloning Options

A large number of Cloning options can be configured. These options are related and interdependent. These options are comprehensive with very complex nested affects.

bool Classes;

Clone classes, sometimes you may not want to clone classes. Then disable this cloning for classes.

bool ProjectUsers;

Clone project users links. This will clone and give users access to the new project. Please be aware of the security risks of this option.

bool Service;

Enable the cloning of all top level services linked to a project. The service and its settings will be cloned.

bool ServiceRecursive;

To enabled cloning of nested/recursive services. This only has an effect if Service cloning is enabled.

bool ServiceDefSettings;

Clone service definitions settings. This should be enabled if service cloning is enabled. This will allow runtime/processing settings to be cloned.

bool ParamDefSettings;

Clone parameter definitions settings.

This allow for custom code and rule settings to be cloned. This should normally be enabled.

List<ParameterDefinitionCategory?> ParamDefCategories;

Limit cloning of parameter definition categories. Normally this is not configured and should only be configured in advanced situations.

bool Ratings;

Clone ratings user gave a service. Does not have any processing impact.

bool Documents;

Enabled cloning of documents. This mus be enabled to for documents to be cloned. Additional setting will apply. See additions document usage settings.

int? DocumentFromId;

For performance, only clone documents from a certain document Id range. This is also helpfull for clong only a certain range of document id's.

int? DocumentToId;

For performance, only clone documents to a certain document Id range. This is also helpfull for clong only a certain range of document id's.

bool? DocumentDescending;

Clone documents in reverse descending order. Normally cloning start with oldest documents first and then move to newer documents. Sometimes you may want to start cloning with newest documents and move back in time to older documents.

List<int> DocumentIds;

Optionally limit document cloning to very specific document Ids. This is used in combination of document usages.

List<int?> DocumentCategoriesSrc;

Only clone documents with of a specific source classification or vendor This is a list of classification ids

List<int?> DocumentCategoriesDst;

Only clone documents with of a specific destination classification or vendor This is a list of classification ids

Dictionary<UsageType, List<string>> SkipDocs;

To bypass or skip documents of a certain usage and filename. Usefull to bypass datasets with certain names... e.g. UsageType.DataSet: ["ExtractionPatterns", "StoreMaster", "Storemaster New", "PurchaseOrdersItems", "PurchaseOrders", "CostCenters", "GLCodes"]

Dictionary<DocumentStatus, List<string>> SkipStatus;

Skip Documents of a certain status and filename. This is similair to the document usage skip list.

int DocumentMaxDegreeOfParallelism;

Process documents in Parallel for better performance performance.

bool DocumentsDeletedState;

Normally documents in a deleted state are not cloned. Used this to enabled cloning of deleted documents. Sometime you may wish to clone documents currently in a "soft delete" state.

bool DocumentsDataSet;

Clone datasets via documents

bool DocumentsInbox;

Enable or disable cloning of inbox documents

bool DocumentsOutBox;

Enable or disable cloning of outbox documents

bool DocumentsDefinitions;

Enable or disable cloning of definition documents

bool DocumentsTraining;

Enable or disable cloning of training documents

bool DocumentsLabelling;

Enable or disable cloning of labelling documents

bool DocumentData;

Enable or disable cloning of documents data. This is also dependant on Documents flags

bool Parameters;

Enable or disable cloning of document parameters. This ia heavy in processing, but all values and parameters on documnets are cloned.

bool Verifications;

Enable or disable cloning of document verifications. This should normally be enabled when parameter cloning is enabled.

bool RedoVerifications;

When verifications are cloned, you may wish to redo verification. This should only be used once off.

bool Work;

Enable cloning of work items related to documents.

bool Audit;

Enable cloning of audit records.

bool Logs;

Enable cloning document processing logs.

bool Transactions;

Enable cloning of transactions. This should normally be disabled. This will copy debit trsansactions and may cause unintended charge to occur.

bool Payments;

Enable cloning of payment records. Can only be selected with Transactions.

bool ProviderProjects;

Services may be executed in the cloud or by remote serice providers. This will clone projects and configurations on provider. This is normally required. e.g. clone MS Forms, Document Intelligence projects on provider

bool ProviderModels;

This will clone models on a provider. This is normally required. e.g. clone MS Forms, Document Intelligence projects on provider

ICloneChecks Checks;

Additional cloning checks to finetune the and allow additional cloning checks.

Cloning Validation Checks

Clone check assist with performance and syncing requirements. These related checks will only be used when the relevant cloning setting is enabled. e.g. ReProcessInboxDocument will only have an effect when inbox document are cloned.

bool ProjectExist;

Check if a project exists before creating a new project. Search by using the name.

bool DoNotCreatePD;

Do not create definitions. Otherwise it will use and extensive search to find a similar parameter definition.

bool ReProcessDocumentDataOnly;

Only reprocess clone document data. Do not reporcess parameters and verifications...

bool ReProcessDataSetDocument;

Enabled reprocessing of Dataset documents. If the document is found, it will skip any further processing. If this setting is enabled, it will retry to process and sync parameters and verifications.

bool ReProcessInboxDocument;

Enabled reprocessing of inbox documents If the document is found, it will skip any further processing. If this setting is enabled, it will retry to process and sync parameters and verifications.

bool ReProcessOutboxDocument;

Enabled reprocessing of outbox documents If the document is found, it will skip any further processing. If this setting is enabled, it will retry to process and sync parameters and verifications.

bool ReProcessDefinitionDocument;

Enabled reprocessing of definitions documents If the document is found, it will skip any further processing. If this setting is enabled, it will retry to process and sync parameters and verifications.

bool ReProcessTrainingDocument;

Enabled reprocessing of training documents If the document is found, it will skip any further processing. If this setting is enabled, it will retry to process and sync parameters and verifications.

bool ReProcessLabellingDocument;

Enabled reprocessing of labelling documents. If the document is found, it will skip any further processing. If this setting is enabled, it will retry to process and sync parameters and verifications.

bool ParameterExists;

Do additional check to see if a parameter already exists.

bool VerificationExists;

Check if a verification parameter already exist.

bool TransactionExists; //by default check - we dont want to duplicate

Additional checks to see if a transaction already exists to avoid duplicate transactions.

bool AuditExists;

Additional checks to see if an audit record already exists to avoid duplicates

bool WorkItemsExists;

Additional checks to see if a work items already exists to avoid duplicates

bool LogsExists;

Additional checks to see if a log record already exists to avoid duplicates

bool PaymentExists;

Additional checks to see if a payment already exists to avoid duplicates By default this check is enabled - we dont want to duplicate payments.