Import Large Watchlist

[Former Member]

I am trying to import and regularly update a large watchlist into the SIEM. The watchlist is ~242k domains. 

I have tried using the SIEM API for this however, SIEM only allows for the addition of ~2K values at a time (WHY!??)   so this is not realistic. 

My next attempt was to setup a NFS share that the SIEM could pull from, but this just causes the SIEM to choke and freeze up. 

Does anyone know of a good way to create a dynamic watchlist that can handle this many values?

[brenta]

For large watchlists I have always done an import using a Cyberthreat feed (TAXII), this is also one of the only ways to do back tracing. I am surprised your ESM had problems with the NFS import, it's always worked sufficiently fine for me.

Do you have a lot of operations on your SIEM?

Could you run `iostat -dhxm 10` and wait about 1 minute then CTRL-C and paste or message me the results?