I am trying to get all the objects in a vault/class into my allFiles variable. For some reason, I get odd results. I should be getting back about 2.4 million records but I am only getting back about 2.2 when I try to just pull everything. When Trying to pull back as in the code below in batches of 500, or 5000 I also get weird results. When using a batch size of 500, I get back ~1 million files. When I try to pull back with the exact same code in batches of 5000, I get back on 750K records. Not sure if I am running into some issue with my pagination or some other limition but doesnt make sense to me.
Here is my code :
public static List<ObjectVersion> GetFileBatchWithConditionsEx(
Vault vault,
IEnumerable<SearchCondition> baseConditions
)
{
const int batchSize = 500;
var allResults = new List<ObjectVersion>();
int lastSeenID = 0;
while (true)
{
var conditions = new SearchConditions();
foreach (var c in baseConditions)
conditions.Add(-1, c);
var idFilter = new SearchCondition
{
ConditionType = MFConditionType.MFConditionTypeGreaterThanOrEqual,
Expression = new Expression
{
DataStatusValueType = MFStatusType.MFStatusTypeObjectID
},
TypedValue = new TypedValue()
};
idFilter.TypedValue.SetValue(MFDataType.MFDatatypeInteger, lastSeenID);
conditions.Add(-1, idFilter);
var results = vault.ObjectSearchOperations
.SearchForObjectsByConditionsEx(
conditions,
MFSearchFlags.MFSearchFlagReturnLatestVisibleVersion,
true, // sort
batchSize, // <-- maxResults
3600 // timeout
);
if (batch.Count == 0)
break; // we’re done
allResults.AddRange(batch);
WaiterService.UpdateStatus($"Fetched {allResults.Count} records so far...");
lastSeenID = batch.Max(o => o.ObjVer.ID) + 1;
}
return allResults;
}
Assume it could be something to do with sorting or something similar...
Ideas ?
Thanks in advance,
Pat
