Need to speed up metadata extraction

I have a project where the client wants to get the metadata out of one of their m-files classes.  Currently, the vault has 1.96 million files.  All I am trying to pull is the metadata for each file.  My tool runs and it was running much faster a few days ago but yesterday it slowed dramatically.  My tool was estimating that it would take about 3 days to get all the metadata and now it is 16 days.  So, my questions are, what causes the change in performance and what are some best practices and maybe COM functions that would make this better.  They will be doing this multiple times a year.  I understand why the download takes some time but not sure if there are some better classes/object to bulk download metadata

My metadata extraction uses the following code

public string GetMetadataAsJson(ObjectVersion objectVersion, Vault vault, string filePath, string className)
{

var metadataDict = new Dictionary<string, object>
{
["MFilesFolderPath"] = filePath,
["ClassName"] = className
};


PropertyValues properties = vault.ObjectPropertyOperations.GetProperties(objectVersion.ObjVer);


var propertyDefsCache = new Dictionary<int, string>();

foreach (PropertyValue propertyValue in properties)
{

if (!propertyDefsCache.TryGetValue(propertyValue.PropertyDef, out string propertyDefName))
{
PropertyDef propertyDef = vault.PropertyDefOperations.GetPropertyDef(propertyValue.PropertyDef);
propertyDefName = propertyDef.Name;
propertyDefsCache[propertyValue.PropertyDef] = propertyDefName;
}


metadataDict[propertyDefName] = propertyValue.Value.DisplayValue;
}

return JsonConvert.SerializeObject(metadataDict, Formatting.Indented);
}

Parents Reply
  • How about this, Make a view that has your files. Remove the view limit, remove the timeout from preferably server or any client and right click and download everything in said view ? Shouldn't take longer than 16 days and you would have the files and the metadata

    or you can go for segmentation for data like, make multiple views based on creation date of the files. And do the export operation

Children
No Data