Mass Delete Files

I had started on a commandline tool for this a while ago, I ended up converting it to a VAF that can be triggered manually or on a schedule. The primary reason I converted it to VAF was for the "Search Condition Editor" that is standard functionality in the VAF API and Admin Tool.  In our use cases, we wanted to destroy objects based on very specific metadata - which is harder to do in command line.  Let me know if you want some code snippets from what we built.

Would you share it for the community?

Would you share it for the community?

Of course! Here's the main portion of it.  I took out a bunch log logging and my-company-specific lines, so there's a chance I missed a semi-colon or something, so don't be surprised if there's a couple immediate errors raised.  The code is wrapped in a Task (or queue, I forget what it's called).  

using DCPMidstream.Common.Configuration;
using DCPMidstream.Common.Logging;
using DCPMidstream.Common.TaskProcessing;
using MFiles.VAF.AppTasks;
using MFiles.VAF.Common;
using MFiles.VAF.Extensions.Dashboards;
using MFilesAPI;
using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Linq;

namespace DCPMidstream.DestroyObjects
{
    public class Module : ModuleWithTaskProcessors
    {
        [TaskProcessor(RootConfig.TaskQueueIdConcurrent, DestroyObjectsConfig.TaskType, TransactionMode = TransactionMode.Unsafe)]
        [ShowOnDashboard(ModuleName, ShowRunCommand = true)]
        public void Process(ITaskProcessingJob<TaskDirective> job)
        {

            Vault vault = job.Vault;

            // Shorthand config and setup logging
            DestroyObjectsConfig config = RootConfig.DestroyObjects;

            // Process enabled each rule
            for (int i = 0; i < config.Rules.Count; i++)
            {
                DestroyRule rule = config.Rules[i];
                if (!rule.Enabled) continue;

                MFSearchBuilder mFSearchBuilder = new MFSearchBuilder(vault, rule.Filter.ToApiObject(vault));

                IEnumerable<int> objIDsPrevious = new List<int>();

                int iteration = 0;
                do
                {
                    int pageSize = RootConfig.DestroyObjects.PageSize;

                    ObjectSearchResults results = mFSearchBuilder.Find(
                      maxResults: pageSize,
                      searchTimeoutInSeconds: 0);

                    if (results.Count == 0)
                    {
                        break;
                    }

                    ObjIDs objIDs = results.GetAsObjectVersions().GetAsObjVers().GetAllDistinctObjIDs();

                    // Check if result count is smaller than page size (getting towards the end)
                    if (results.Count < pageSize)
                    {
                        // If list of Obj IDs is the same as the last result set, break - these are undeletables
                        if (results.GetAsObjectVersions().GetAsObjVers().GetAllDistinctObjIDs().Cast<ObjID>().Select(x => x.ID).Except(objIDsPrevious).Count() == 0)
                        {
                            break;
                        }
                        // If the list is not the same as the previous one, this is the first set with less results than page size - log IDs to be checked next page
                        else
                        {
                            objIDsPrevious = objIDs.Cast<ObjID>().Select(x => x.ID);
                        }
                    }

                    // Kill switch - if the application or rule is disabled at any time, it will break the loop and stop processing this rule
                    if (!RootConfig.DestroyObjects.Enabled || !RootConfig.DestroyObjects.Rules[i].Enabled)
                    {
                        break;
                    }

                    try
                    {
                        vault.ObjectOperations.DestroyObjects(objIDs);
                    }
                    catch (Exception e)
                    {
                        // Add logging
                    }
                } while (true);
            }
        }
    }
}

Here's the Config that the code uses

using MFiles.VAF.Configuration;
using MFiles.VAF.Configuration.JsonAdaptor;
using MFiles.VAF.Extensions;
using System.Collections.Generic;
using System.Runtime.Serialization;

namespace DCPMidstream.Common.Configuration
{
    [DataContract]
    public class DestroyObjectsConfig
    {
        public const string TaskType = "DestroyObjects";

        [DataMember]
        [RecurringOperationConfiguration(RootConfig.TaskQueueIdConcurrent, TaskType,
            IsRequired = true,
            HelpText = "Determines how often the destroy process runs.")]
        public Frequency Schedule { get; set; }

        [DataMember]
        [JsonConfIntegerEditor(
            Label = "Page Size",
            HelpText = "Determines how many object to search for at once.  Searching will continue in batches until 0 results are returned.\n\nM-Files API currently has a limitation where you cannot destroy more than 500 objects at one time.",
            Min = 1,
            Max = 500,
            DefaultValue = 100)]
        public int PageSize { get; set; } = 100;

        [DataMember]
        [JsonConfEditor(
            IsRequired = true,
            HelpText = "When the indicated object is created new objects will be created.",
            ChildName = "Rule")]
        public List<DestroyRule> Rules { get; set; }
    }

    [DataContract]
    public class DestroyRule
    {
        [DataMember]
        [JsonConfEditor(
            IsRequired = true,
            HelpText = "A user-friendly description to help people find it later.")]
        public string Name { get; set; }

        [DataMember]
        [JsonConfEditor(
            DefaultValue = false)]
        public bool Enabled { get; set; }

        [DataMember]
        [JsonConfEditor(
            IsRequired = true,
            HelpText = "This rule will apply to any objects that satisfy these search conditions.")]
        public SearchConditionsJA Filter { get; set; }
    }
}

The best part of the code (IMHO) is the property in the config called Filter of type SearchConditionsJA.  That's where all the magic happens.  It's presents a brilliant UI in admin tool for easy search configuration and then then if you look in the main code, there's one line of code:

MFSearchBuilder mFSearchBuilder = new MFSearchBuilder(vault, rule.Filter.ToApiObject(vault));

and it returns everything that matches what you configured.

The other nice part is the property called Schedule of type Frequency.  It let's the admin decide how often the Task should run.  Or, since I specified "[ShowOnDashboard(ModuleName, ShowRunCommand = true)]" on the method that runs as a task (first block of code), the process can also be triggered manually at any time from the admin tool dashboard.

You may not be able to run this in your vault as is, but hopefully it gives you some ideas on how you can do something like it in your vault.  Good luck! Let me know if I can help.