주문 관련 배치 작업 중 변경감지 안되서 정리함. 현재는 Spring batch 에서 reader processor writer step 중 ItemReader 내부에서 last read id + offset size 로 Pagination 해서 읽어오고 있음. 그래야 mysql offset based 로 pagination 해서 읽었을 때 오는 오버헤드를 확 줄일 수 있음. index 타고 거기서부터 offset 읽으면 빠르기 때문. 일단 여기서 이 ItemReader 는 영속성에 들어있는 엔티티를 detached 시킨다고함. 그러니까 reader step 지나면 chunk 내에서 다시 merge 시켜줘야함. 영속성 다 관리할려면 메모리가 힘드니까 중간에 detached 시킨다고 한다.

ItemReader for reading database records built on top of JPA.

It executes the JPQL setQueryString(String) to retrieve requested data. The query is executed using paged requests of a size specified in AbstractPagingItemReader.setPageSize(int). Additional pages are requested when needed as AbstractItemCountingItemStreamItemReader.read() method is called, returning an object corresponding to current position.

The performance of the paging depends on the JPA implementation and its use of database specific features to limit the number of returned rows.

Setting a fairly large page size and using a commit interval that matches the page size should provide better performance.

In order to reduce the memory usage for large results the persistence context is flushed and cleared after each page is read. This causes any entities read to be detached. If you make changes to the entities and want the changes persisted then you must explicitly merge the entities.

I wrote this down because change detection did not work during an order-related batch job. Currently, in Spring Batch, inside the ItemReader of the reader-processor-writer step, data is read with pagination using last read id + offset size. This greatly reduces the overhead that appears when reading with MySQL offset-based pagination. It is faster because it uses the index and reads from that point. Here, this ItemReader is said to detach entities that are inside the persistence context. So after the reader step, entities inside the chunk need to be merged again. Since managing all persistence state would be hard on memory, it detaches them in the middle.

ItemReader for reading database records built on top of JPA.

It executes the JPQL setQueryString(String) to retrieve requested data. The query is executed using paged requests of a size specified in AbstractPagingItemReader.setPageSize(int). Additional pages are requested when needed as AbstractItemCountingItemStreamItemReader.read() method is called, returning an object corresponding to current position.

The performance of the paging depends on the JPA implementation and its use of database specific features to limit the number of returned rows.

Setting a fairly large page size and using a commit interval that matches the page size should provide better performance.

In order to reduce the memory usage for large results the persistence context is flushed and cleared after each page is read. This causes any entities read to be detached. If you make changes to the entities and want the changes persisted then you must explicitly merge the entities.