Exporting capture histories from the ARW for MARK

Hi everyone,

I am part of the fire salamander research team at Bielefeld University, my research concerns the fire salamander population in the Botanical Garden Bielefeld.

I have a question regarding exporting my data from the Amphibian and Reptile Wildbook (ARW):

I am using the encounter search do download capture histories for MARK (with the individual search I always have the problem that I cannot export our Individual IDs, instead I only get the internal IDs wildbook is using). My problem is that wildbook seems to limit the number of capture sessions. I want to download all data from our project in the Botanical Garden Bielefeld (these are two projects in the wildbook: Botanical Garden Bielefeld Adults & Botanical Garden Bielefeld Adults 2024) of the last two years (29.06.2022 - 29.06.2024) which concludes in a total number of 166 monitoring events.
Is there a way to download all the data I require in one data export? Splitting the download and then pasting the corresponding files together would cause me a lot of extra work. There are many individuals which we encountered throughout the whole 2 years.

And I also have a second question:

Fire salamanders are a nocturnal species and therefore our population monitorings often extend midnight causing us having many encounters in the ARW which time stamps are after midnight. When exporting and handling the data this causes some problems, especially in cases where we conducted two monitorings on consecutive days. The date-based search filters in wildbook are incapable to differentiate whether we encounterd an individual during the first monitoring after midnight or on the second monitoring before midnight since both occasions have the same date. Therefore after downloading the capture histories for MARK I have to control and correct all encounters after midnight, if we also conducted a monitoring on the following day. Since this actually happens quite often, I wanted to ask whether it is possible to either change the border between dates (using noon instead of midnight, this would solve the problem completely, I think) or use time-based search filters instead of date-based (there I could manually set start and end of each capture event and this would solve the problem too).

I hope you can help me.

Thanks a lot in advance. :slight_smile:

Greetings from Germany

Sean

1 Like

Hi @Sean.Grond

I’m going to update this to a feature request since it doesn’t appear that selecting Include marked individual ID as a comment at the end of each line on MARK exports includes the user-facing Individual ID and there isn’t currently a way to filter searches by time, only the date.

I know that improvements to the search pages on Wildbook are on the horizon. As those roll out, it will be easier for us to iterate on things like additional search filters.

Hi @Anastasia,

thanks. And is there something you can do regarding the maximum number of capturing sessions when downloading the MARK data? Or do I have to split it and download the data in multiple sub-sets?

Greets

I’ll see what I can find out for you today.

Can you clarify what you mean about the limit on the capture events? In our testing, we can see that it can allow for 166 events in an export.

Hi,
that is odd. It did not work for me … I’ll try again today

export request too large

after submitting the starting and end dates of all capture sessions, I receive this error

Thanks for letting me know. I need more time too look into this.

I haven’t forgotten about this! I just wanted to let you know I’ll check in next week with an update.

Hi, no problem, no need to hurry. Do you need any of our data (e.g. the dates of the capture sessions) for testing? I can send them to you.

Hi @Sean.Grond,

Do you mind trying again?

If it works, a change I made is effective!

If you get the same (or a different) error, can you please copy and paste the long URL? This will allow me to exactly test on our side.

Thank you,
Jason

Hi @jason,

I tried again and now received this error.

This is the URL you were asking for:
https://amphibian-reptile.wildbook.org/MarkRecaptureEncounters?locationCodeField=BGBi&datepicker1=2022-06-29&datepicker2=2024-06-30&male=male&female=female&unknown=unknown&alive=alive&dead=dead&numSearchedObs=1&lifeStageField=None&measurementsize(operator)=gteq&label0=None&numResights=1&satelliteTagName=None&biomeasurement13C(operator)=gteq&biomeasurement15N(operator)=gteq&biomeasurement34S(operator)=gteq&organizationId=None&projectId=c14362c3-4de5-46cb-b500-d99c367f5902&submitSearch=Search+Encounters&encounterExport=true&numberSessions=166&includeQueryComments=true&includeIndividualID=true&datepicker0start=2022-06-30&datepicker0end=2022-07-01&datepicker1start=2022-07-06&datepicker1end=2022-07-07&datepicker2start=2022-07-07&datepicker2end=2022-07-08&datepicker3start=2022-07-21&datepicker3end=2022-07-22&datepicker4start=2022-07-26&datepicker4end=2022-07-27&datepicker5start=2022-08-15&datepicker5end=2022-08-16&datepicker6start=2022-08-22&datepicker6end=2022-08-23&datepicker7start=2022-08-27&datepicker7end=2022-08-28&datepicker8start=2022-09-08&datepicker8end=2022-09-09&datepicker9start=2022-09-09&datepicker9end=2022-09-10&datepicker10start=2022-09-10&datepicker10end=2022-09-11&datepicker11start=2022-09-14&datepicker11end=2022-09-15&datepicker12start=2022-09-16&datepicker12end=2022-09-17&datepicker13start=2022-09-20&datepicker13end=2022-09-21&datepicker14start=2022-09-24&datepicker14end=2022-09-25&datepicker15start=2022-09-25&datepicker15end=2022-09-26&datepicker16start=2022-09-26&datepicker16end=2022-09-27&datepicker17start=2022-09-28&datepicker17end=2022-09-29&datepicker18start=2022-09-30&datepicker18end=2022-10-01&datepicker19start=2022-10-01&datepicker19end=2022-10-02&datepicker20start=2022-10-02&datepicker20end=2022-10-03&datepicker21start=2022-10-04&datepicker21end=2022-10-05&datepicker22start=2022-10-13&datepicker22end=2022-10-14&datepicker23start=2022-10-14&datepicker23end=2022-10-15&datepicker24start=2022-10-15&datepicker24end=2022-10-16&datepicker25start=2022-10-24&datepicker25end=2022-10-25&datepicker26start=2022-11-03&datepicker26end=2022-11-04&datepicker27start=2022-11-16&datepicker27end=2022-11-17&datepicker28start=2022-11-17&datepicker28end=2022-11-18&datepicker29start=2022-11-24&datepicker29end=2022-11-25&datepicker30start=2023-01-04&datepicker30end=2023-01-05&datepicker31start=2023-01-14&datepicker31end=2023-01-15&datepicker32start=2023-02-16&datepicker32end=2023-02-17&datepicker33start=2023-02-17&datepicker33end=2023-02-18&datepicker34start=2023-02-18&datepicker34end=2023-02-19&datepicker35start=2023-02-20&datepicker35end=2023-02-21&datepicker36start=2023-03-13&datepicker36end=2023-03-14&datepicker37start=2023-03-14&datepicker37end=2023-03-15&datepicker38start=2023-03-16&datepicker38end=2023-03-17&datepicker39start=2023-03-21&datepicker39end=2023-03-22&datepicker40start=2023-03-22&datepicker40end=2023-03-23&datepicker41start=2023-03-23&datepicker41end=2023-03-24&datepicker42start=2023-03-24&datepicker42end=2023-03-25&datepicker43start=2023-03-25&datepicker43end=2023-03-26&datepicker44start=2023-03-30&datepicker44end=2023-03-31&datepicker45start=2023-03-31&datepicker45end=2023-04-01&datepicker46start=2023-04-11&datepicker46end=2023-04-12&datepicker47start=2023-04-12&datepicker47end=2023-04-13&datepicker48start=2023-04-15&datepicker48end=2023-04-16&datepicker49start=2023-04-22&datepicker49end=2023-04-23&datepicker50start=2023-04-23&datepicker50end=2023-04-24&datepicker51start=2023-04-24&datepicker51end=2023-04-25&datepicker52start=2023-04-28&datepicker52end=2023-04-29&datepicker53start=2023-05-01&datepicker53end=2023-05-02&datepicker54start=2023-05-04&datepicker54end=2023-05-05&datepicker55start=2023-05-05&datepicker55end=2023-05-06&datepicker56start=2023-05-09&datepicker56end=2023-05-10&datepicker57start=2023-05-10&datepicker57end=2023-05-11&datepicker58start=2023-05-15&datepicker58end=2023-05-16&datepicker59start=2023-05-22&datepicker59end=2023-05-23&datepicker60start=2023-05-23&datepicker60end=2023-05-24&datepicker61start=2023-06-06&datepicker61end=2023-06-07&datepicker62start=2023-06-20&datepicker62end=2023-06-21&datepicker63start=2023-06-22&datepicker63end=2023-06-23&datepicker64start=2023-06-29&datepicker64end=2023-06-30&datepicker65start=2023-07-01&datepicker65end=2023-07-02&datepicker66start=2023-07-05&datepicker66end=2023-07-06&datepicker67start=2023-07-09&datepicker67end=2023-07-10&datepicker68start=2023-07-10&datepicker68end=2023-07-11&datepicker69start=2023-07-13&datepicker69end=2023-07-14&datepicker70start=2023-07-15&datepicker70end=2023-07-16&datepicker71start=2023-07-17&datepicker71end=2023-07-18&datepicker72start=2023-07-19&datepicker72end=2023-07-20&datepicker73start=2023-07-21&datepicker73end=2023-07-22&datepicker74start=2023-07-27&datepicker74end=2023-07-28&datepicker75start=2023-07-31&datepicker75end=2023-08-01&datepicker76start=2023-08-03&datepicker76end=2023-08-04&datepicker77start=2023-08-05&datepicker77end=2023-08-06&datepicker78start=2023-08-19&datepicker78end=2023-08-20&datepicker79start=2023-08-24&datepicker79end=2023-08-25&datepicker80start=2023-08-30&datepicker80end=2023-08-31&datepicker81start=2023-08-31&datepicker81end=2023-09-01&datepicker82start=2023-09-01&datepicker82end=2023-09-02&datepicker83start=2023-09-09&datepicker83end=2023-09-10&datepicker84start=2023-09-12&datepicker84end=2023-09-13&datepicker85start=2023-09-13&datepicker85end=2023-09-14&datepicker86start=2023-09-15&datepicker86end=2023-09-16&datepicker87start=2023-09-18&datepicker87end=2023-09-19&datepicker88start=2023-09-19&datepicker88end=2023-09-20&datepicker89start=2023-09-21&datepicker89end=2023-09-22&datepicker90start=2023-09-23&datepicker90end=2023-09-24&datepicker91start=2023-09-29&datepicker91end=2023-09-30&datepicker92start=2023-10-05&datepicker92end=2023-10-06&datepicker93start=2023-10-08&datepicker93end=2023-10-09&datepicker94start=2023-10-09&datepicker94end=2023-10-10&datepicker95start=2023-10-12&datepicker95end=2023-10-13&datepicker96start=2023-10-14&datepicker96end=2023-10-15&datepicker97start=2023-10-15&datepicker97end=2023-10-16&datepicker98start=2023-10-19&datepicker98end=2023-10-20&datepicker99start=2023-10-20&datepicker99end=2023-10-21&datepicker100start=2023-10-23&datepicker100end=2023-10-24&datepicker101start=2023-10-24&datepicker101end=2023-10-25&datepicker102start=2023-10-26&datepicker102end=2023-10-27&datepicker103start=2023-10-28&datepicker103end=2023-10-29&datepicker104start=2023-10-29&datepicker104end=2023-10-30&datepicker105start=2023-11-04&datepicker105end=2023-11-05&datepicker106start=2023-11-05&datepicker106end=2023-11-06&datepicker107start=2023-11-06&datepicker107end=2023-11-07&datepicker108start=2023-11-09&datepicker108end=2023-11-10&datepicker109start=2023-11-10&datepicker109end=2023-11-11&datepicker110start=2023-11-13&datepicker110end=2023-11-14&datepicker111start=2023-11-14&datepicker111end=2023-11-15&datepicker112start=2023-11-15&datepicker112end=2023-11-16&datepicker113start=2023-11-19&datepicker113end=2023-11-20&datepicker114start=2023-11-23&datepicker114end=2023-11-24&datepicker115start=2023-11-30&datepicker115end=2023-12-01&datepicker116start=2023-12-06&datepicker116end=2023-12-07&datepicker117start=2023-12-10&datepicker117end=2023-12-11&datepicker118start=2023-12-15&datepicker118end=2023-12-16&datepicker119start=2023-12-17&datepicker119end=2023-12-18&datepicker120start=2023-12-19&datepicker120end=2023-12-20&datepicker121start=2023-12-20&datepicker121end=2023-12-21&datepicker122start=2023-12-22&datepicker122end=2023-12-23&datepicker123start=2023-12-29&datepicker123end=2023-12-30&datepicker124start=2024-01-05&datepicker124end=2024-01-06&datepicker125start=2024-01-13&datepicker125end=2024-01-14&datepicker126start=2024-01-21&datepicker126end=2024-01-22&datepicker127start=2024-01-26&datepicker127end=2024-01-27&datepicker128start=2024-02-03&datepicker128end=2024-02-04&datepicker129start=2024-02-04&datepicker129end=2024-02-05&datepicker130start=2024-02-09&datepicker130end=2024-02-10&datepicker131start=2024-02-14&datepicker131end=2024-02-15&datepicker132start=2024-02-20&datepicker132end=2024-02-21&datepicker133start=2024-02-24&datepicker133end=2024-02-25&datepicker134start=2024-02-29&datepicker134end=2024-03-01&datepicker135start=2024-03-05&datepicker135end=2024-03-06&datepicker136start=2024-03-08&datepicker136end=2024-03-09&datepicker137start=2024-03-11&datepicker137end=2024-03-12&datepicker138start=2024-03-13&datepicker138end=2024-03-14&datepicker139start=2024-03-15&datepicker139end=2024-03-16&datepicker140start=2024-03-22&datepicker140end=2024-03-23&datepicker141start=2024-03-23&datepicker141end=2024-03-24&datepicker142start=2024-03-24&datepicker142end=2024-03-25&datepicker143start=2024-03-27&datepicker143end=2024-03-28&datepicker144start=2024-04-01&datepicker144end=2024-04-02&datepicker145start=2024-04-02&datepicker145end=2024-04-03&datepicker146start=2024-04-03&datepicker146end=2024-04-04&datepicker147start=2024-04-04&datepicker147end=2024-04-05&datepicker148start=2024-04-11&datepicker148end=2024-04-12&datepicker149start=2024-04-16&datepicker149end=2024-04-17&datepicker150start=2024-04-19&datepicker150end=2024-04-20&datepicker151start=2024-04-24&datepicker151end=2024-04-25&datepicker152start=2024-05-02&datepicker152end=2024-05-03&datepicker153start=2024-05-03&datepicker153end=2024-05-04&datepicker154start=2024-05-04&datepicker154end=2024-05-05&datepicker155start=2024-05-09&datepicker155end=2024-05-10&datepicker156start=2024-05-16&datepicker156end=2024-05-17&datepicker157start=2024-05-18&datepicker157end=2024-05-19&datepicker158start=2024-05-21&datepicker158end=2024-05-22&datepicker159start=2024-05-24&datepicker159end=2024-05-25&datepicker160start=2024-05-29&datepicker160end=2024-05-30&datepicker161start=2024-06-06&datepicker161end=2024-06-07&datepicker162start=2024-06-10&datepicker162end=2024-06-11&datepicker163start=2024-06-17&datepicker163end=2024-06-18&datepicker164start=2024-06-18&datepicker164end=2024-06-19&datepicker165start=2024-06-29&datepicker165end=2024-06-30

Thanks a lot!
Sean

Hi @Sean.Grond

I increased the allowed URL size, and I was able to down the imp file. Can you please try again?

Thank you,
Jason

Hi @jason,

thanks! I’ll try again today and will let you know whether it worked.

It worked! I have my data.
Thanks a lot @Anastasia & @jason!

:upside_down_face:

2 Likes

Great; thanks for letting us know @Sean.Grond!

I’ll leave this feature request open to cover the remaining issues you raised:

Hi everyone,

I would like to export the data of my 38 nocturnal monitoring events (in ARW at the export section as a .inp file).

I recently came across the post by Sean.Grond discussing the issue of nocturnal species encounters and the complications caused by date-based search filters when exporting data. Sean mentioned the changing of the boundary between dates (using noon instead of midnight) or using time-based search filter.

I have a similar/ the same question and wanted to ask if you can explain me again whether these changes are possible.

Thank you all very much for your support!

Best regards,
Felix Stumpf.

The changeover from one day to the next occurring at midnight is not likely to change. This meets international standards for reporting date and time.

This is still an open feature request, so there’s an opportunity for discussion regarding filtering encounters by time.

Hi @jason,

I noticed something else in my exported data. There seem to be some individuals missing.
The downloaded IMP file contains exactly 7,600 capture histories but in total I have over 10,000 individuals in the corresponding projects.

Could it be that the IMP download has a limit of how many capture histories can be exported? That would explain why it is such an even number of capture histories in my file.

Regards

Sean

Hi @Sean.Grond

Can you share an example of an Encounter of an individual that should fall into one of the sessions but isn’t?

Thanks,
Jason