Can /structure/objecttypes/(type)/refreshstatus be called concurrently?

I believe if one refresh is already ongoing then the subsequent call will be ignored.

Hi Craig, would you be able to confirm that? 

Also, assuming it does the ignore.. Are there any events that get firred when it finishes the refresh or updates the value list?

E.x.: MFEventHandlerAfterRefreshValueList

E.x.: MFEventHandlerAfterExternalSourceRefresh

Feature Request: I think MFEventHandlerAfterExternalSourceRefresh should be added as an event if it doesn't exist

I have queried this from our product team.  I will raise a tracker item for these events later today.

Hi Craig, thank you!

I had a response from the product team that subsequent calls will queue up a further refresh when the current one completes, but there can only be a maximum of one in that queue. 

What I mean is consider a situation where a refresh takes five minutes. If you ask for a refresh to be done after 60s, then again after 60s, then again after 60s, the system will queue up a single refresh for after the current one completed.

You would probably want to think about this a little though. Depending upon the frequency of these events you could get into a situation where the system is almost constantly refreshing.

I have created a tracker item for this improvement request: 169127.

Can you give me some details on what exactly you would like to do in this event? What would you like to do when the refresh has occurred? 

Hi Craig,

Assuming the EventHandlerEnvironment class as a ValueListId property or some way of getting the ID of the value list that was changed, we'd use that do the following

1. pull in all the value list items for that value list
2. compare that with what's in the database
3. Kick off a refresh if the value list is still out of sync (quick refresh if it's only missing new data, otherwise full refresh)

NOTE: We're doing alll this logic outside of the vault. We currently have a vault application that publishes/forwards MFEventHandler events to our AWS event bridge, so we would doing this pull & compare logic inside of an AWS Lambda that will use MFile's REST API to pull in the value list items.

This can all be avoided if M-Files can smartly queue the refresh requests because we control the "External Databases" that are being used to populate the ValueList and ObjectTypes in M-Files, so we'd like to proactively refresh the value list items when know M-Files will be out of sync with thet database.

We would trigger a quick refresh for inserts and full refresh for udates and deletes.

When I say smartly, I mean it can do things like

• Skip all pending refreshes for a particular value list or object type that were requested before the currently running full refresh for the same value list/object type was started... but only delete it if the refresh was successful.
• Skip all pending quick refreshes for a particular value list or object type that were requested before the currently running quick refresh for the same value list/object type was started... but only delete it if the refresh was successful.

I'm not sure I really understand.  It may be worth scheduling some time with your partner (if you have one, or via a local technical team in your region if not) to go through this in more detail.  It sounds rather an awkward approach right now.

For example:

• I don't understand why you would wait for the refresh to finish, then kick off another refresh.  At this point the data should be largely correct, surely?
• The "smart refresh"; I'm not sure what you mean here.  I assume you want to basically skip any synchronisation requests that occur provided the currently-running one completes successfully; is that correct?

Honestly: if you want 100% complete and full control over the data synchronisation process, especially if you have lots of systems involved, it could be that the correct course of action is to avoid the external object type data source and have the third-party system (EventBridge) be in complete control of pushing items into the appropriate value lists via the APIs.  It can react when single items are created, modified, or deleted in the source system, and then be in control of ensuring that the appropriate action is taken in M-Files.  It can even do this down to the single-item-level (e.g. one item is added in a list of a million; this can simply add the one, rather than rely on the sync process to iterate over all that redundant data).

This isn't an approach that I push a lot, but it does simplify things in less common situations where there's huge data volumes or other such considerations.

Hi Craig,

I mentioned that to address the race condition of the data in the external data provider changing while the refresh is in progress (E.x: What happens in if the data changes right before the refresh process ends).

Also, I still need the MFEventHandlerAfterExternalSourceRefresh event with the ValueListId in order to effeciently take full control over the data synchronisation process.

E.x.: I don't know when the refresh is complete, so I'm just going to keep pulling the value lists in a loop until the data is in sync.