Options
  • State Suggested Answer
  • Replies 1 reply
  • Answers 1 answer
  • Subscribers 10 subscribers
  • Views 32 views
  • Users 0 members are here
Related

Paging through items in a vault

darren.oakey

I have a need to iterate over every item in one of our vaults, and I can't find a reliable way of doing it.

The first thing I thought of was just to read by id - for instance read id's 1-1000, then 1000-2000 and so on - so I tried /objects/0?q=id>=1000&id<=1099&limit=100.     And I got back 100 items - but they just had random ids

I read a lot more and found the "p" syntax - and looked for the id of the id property - but there didn't seem to be one - so I thought - I'll use created date time.  The created field in /structure/properties was 20 - so I tried this:

/objects/0?q=p20>=2000-01-01T00:00:00Z&limit = 100

now - I know _everything in the database_ is later than 2000 - but I got back nothing.

What am I doing wrong?   And what is the best way to achieve what I want?   Basically I want to loop through every single item (there may be about 1,000,000 of them) - in a repeatable way such that I can stop at any point and then start again from where I left off with no risk of missing anything.   And then run it later and pick up any new values.  

How can I do that?

Thanks,

Darren

Parents
  • 0

    Pagination of search results is currently painful, although there are ongoing discussions around how we remove some of this pain.  I hope to have some news on this in the coming months.  With the COM API you can definitely query by ID (we have the concept of a "segmented search"); with the REST API it is also possible but uses an undocumented parameter so is not something I would personally recommend for a production environment.

    It looks like your query for the datetime search was incorrect.  I think you should not have the "q=" at the start of the querystring; I think it's trying to search by that datetime string and that's why nothing was returned.

Reply
  • 0

    Pagination of search results is currently painful, although there are ongoing discussions around how we remove some of this pain.  I hope to have some news on this in the coming months.  With the COM API you can definitely query by ID (we have the concept of a "segmented search"); with the REST API it is also possible but uses an undocumented parameter so is not something I would personally recommend for a production environment.

    It looks like your query for the datetime search was incorrect.  I think you should not have the "q=" at the start of the querystring; I think it's trying to search by that datetime string and that's why nothing was returned.

Children
No Data
M-Files
M-Files Support


© 2024 M-Files, All Rights Reserved.