Sudipt
I think as of now to process a huge number of records data sets is the only optimal way

Sudipt
Before suggesting an approach, can you get us the below information?

• Is this a on-time run or an on-going (daily inject) requirement?
• Do you have Read and Write access to external DB or just read-only?
• You have all the information to create Work Object in your external DB table or you need to copy the data, do manipulations inside Pega, and create Work objects?
• In which environment you are trying this in Pega (DEV/QA/UAT/PROD)?

Frequency will be bi weekly or Monthly.
Read access is there .Write can also be given but yet to check on that part.
If the case can be created with external DB data then no need to store it in pega table.
Right now in dev env.