Review M-Files on G2 & get a $25 gift card Arrow right Review us!

Options
  • Locked Locked
  • Replies 6 replies
  • Subscribers 11 subscribers
  • Views 835 views
  • Users 0 members are here
Related
This discussion has been locked.
You can no longer post new replies to this discussion. If you have a question you can start a new discussion

M-Files as a storage for frequent big documents

dejan
Hi there,

I was approached today with a requirement to save big technical reports containing picture snippets and summaries. These reports are usually between 250-500MB each and they are in PDF format.
Another point is that these reports could easily be produce 1-10 times per month.
So if I calculate a middle case 5 documents of 300 MB per month I coming to 60 documents with size 17 GB.
I have did some load test with 200-250MB files over REST based API and it was not particularly fast (obviously due to size of a document). I am currently not aware about complexity of metadata for that specific document type. This can of course influence uploading speed as well.
I am not aware on any document size limit in M-Files but wondering if M-Files is a proper solution for such a use case.
I am also worried because Vault will grow to be quite big and this could influence other document types used in a system (planned 50-100 document types).

What are your experiences with big frequent documents? I would appreciate your experiences.

Dejan
Parents
  • The number of large files is growing by approx 2 pr working day.

    None of the large files are associated with Preview modules, I can't say for sure how they would respond. However, it will take some time for the client to download the content and generate the preview - in particular if the whole file needs to get downloaded before the preview can be generated. On small files I get the impression that it takes more time to start the application then to generate the preview, but that would likely be the other way around on large files.

    The server is a virtual machine in MS Azure (not M-Files Cloud). Capacity is monitored and expanded as the need grows. Search engine is DTsearch. Database is SQL.
    The databases and temp-files are placed on SSD, file storage is on HDD. I do not have details on disk throughput. Currently the server has 32 GB RAM. It will need more in the near future. Free space on file storage needs to be a fair bit larger than the largest file expected to become uploaded. We had a case at one time where the disk filled up with about 90 GB of temporary data while generating one of those large ZIP files which should contain about 40 GB of data. The process could not complete and started over and over again placing a heavy load on the server. You could actually monitor in Explorer on the server how free space on the drive filled up over a matter of a some minutes and then returned to about 90 GB. About 20 minutes later the process started over again in the next attempt. Amazingly the server did not halt and users only noticed a slightly slower response while this was going on.
Reply
  • The number of large files is growing by approx 2 pr working day.

    None of the large files are associated with Preview modules, I can't say for sure how they would respond. However, it will take some time for the client to download the content and generate the preview - in particular if the whole file needs to get downloaded before the preview can be generated. On small files I get the impression that it takes more time to start the application then to generate the preview, but that would likely be the other way around on large files.

    The server is a virtual machine in MS Azure (not M-Files Cloud). Capacity is monitored and expanded as the need grows. Search engine is DTsearch. Database is SQL.
    The databases and temp-files are placed on SSD, file storage is on HDD. I do not have details on disk throughput. Currently the server has 32 GB RAM. It will need more in the near future. Free space on file storage needs to be a fair bit larger than the largest file expected to become uploaded. We had a case at one time where the disk filled up with about 90 GB of temporary data while generating one of those large ZIP files which should contain about 40 GB of data. The process could not complete and started over and over again placing a heavy load on the server. You could actually monitor in Explorer on the server how free space on the drive filled up over a matter of a some minutes and then returned to about 90 GB. About 20 minutes later the process started over again in the next attempt. Amazingly the server did not halt and users only noticed a slightly slower response while this was going on.
Children
No Data
M-Files
M-Files Support


© 2024 M-Files, All Rights Reserved.