Where exactly virus scanning is happening when a file uploaded via rest API. Could you someone help on this?

Have you enabled the registry settings mentioned in the user guide under Antimalware support (on-premises only)?

Based on that guidance I also think that AMSI needs to be enabled ("you must use an anti-virus software that is compatible with Windows Antimalware Scan Interface (AMSI)"), although this is not an area I'm deeply familiar with. If you need to verify this I recommend contacting M-Files Support.

Yes, AMSI need to be enabled and its under investigation, currently its not.
Before that we are trying to find where the files are processing (App server directory or Proxy directory?) when a file uploaded via rest API. As we have Defender AV already exist in servers and Realtime scanning is enabled, there must be some OS level scanning happens before the file processing to Azure blob file data. Unfortunately we are unable to find the exact temp directory in server where M-Files temporary storing and processing.

If you are looking for where temp file is located, I'd go and feed a file that is guaranteed to raise an alarm (preferably, false positive :) ) via rest API.. 

However, as this will raise alarm in anti-virus/anti-malware system, there is a good chance the M-Files and IIS executables will be quarantined and this most probably will comprimise IIS and M-Files operations. So, if I'm gonna do it, I'd do it in a test environment on off-hours :) 

In our test environment i enabled the Antimalware scan in our Application servers as below.
second registry key not enabled, as we know our AV configured in a correct way.

"EnableAntimalwareScanner"=dword:00000001”
"TreatAntimalwareScannerErrorsAsTransferBlockingErrors"=dword:00000000”

Then, i uploaded a EICAR.txt (Industry standard virus file for test file)  via rest API and got response that "Upload session is not found" and in server defender already quarantined the same.

I tried to find the temp location where the detection happen with help of procmon tool, unfortunately unable to spot.
But in AV report its processed and detected from M-Files installation directory & process.

i have not tested this without AMSI, but need to verify this as well to know how it works.

So no path of the file as it shows the path of blocked executables... I suppose you already checked the usual temp paths ? C:\Windows\Temp\.MFilesApplicationTemp and IIS temp ?

Yes, no evidence that the file is detected from any temp path.
Defender detects below M-Files directory alone before it executing. Another thing is that via rest API, Desktop client, web client, The detection/quarantine happens in Application server.

Process: [XXX] MFServer.exe: "mfserver.exe" /SCM (Company: M-Files Corporation, Path: C:\Program Files\M-Files\"Version"\Bin\x64\MFServer.exe) was prevented from executing malicious code 

Maybe the transport between web(proxy) and application server is not encrypted so the file gets detected by network watch component ? 

I can confirm that during M-Files document processing time in Application server the detection is happening and blocking the same.

How i confirmed as below.
i have updated below registry value

"EnableAntimalwareScanner"=dword:00000001”
"TreatAntimalwareScannerErrorsAsTransferBlockingErrors"=dword:00000000”

As below in application server & MFServer services restarted, meaning that antimalware scanning disabled. 

"EnableAntimalwareScanner"=dword:00000000”
"TreatAntimalwareScannerErrorsAsTransferBlockingErrors"=dword:00000000”

Then i again uploaded a EICAR.txt file via rest API and its getting uploaded to the vault successfully.