Spring Batch 프로젝트 생성하기 - Spring Boot 3.1

개요

Spring Boot가 3.0이 출시되면서 Batch도 조금 바뀌게 되어 프로젝트를 생성 및 간단한 실행을 수행해보고자 합니다.

 

 

Gradle Dependencies

dependencies {
	implementation("org.springframework.boot:spring-boot-starter-batch")
	runtimeOnly("org.postgresql:postgresql:42.3.5")
	testImplementation("org.springframework.boot:spring-boot-starter-test")
	testImplementation("org.springframework.batch:spring-batch-test")
	implementation(project(":core"))
}

 

SpringBootApplication

//@EnableBatchProcessing 넣으면 안됨
//@EnableBatchProcessing
@SpringBootApplication
class SpringBatchApplication

fun main(args: Array<String>) {
	runApplication<SpringBatchApplication>(*args)
}

Spring Batch 5.0부터는 @EnableBatchProcessing이 필요하지 않습니다.

@EnableBatchProcessing이 들어가면 오히려 동작하지 않습니다.

 

application.yml

spring:
  datasource:
    url: jdbc:postgresql://localhost:5432/your_database_name
    username: your_postgres_username
    password: your_postgres_password
  batch:
    jdbc:
      initialize-schema: always
      schema: classpath:org/springframework/batch/core/schema-postgresql.sql
    job:
      enabled: true
  jpa:
    hibernate:
      ddl-auto: create-drop
    properties:
      hibernate:
        dialect: org.hibernate.dialect.PostgreSQLDialect

datasource 설정과 batch 설정을 수행해 둡니다.

Batch의 특성 중 하나로 Job이 실행되면서 Meta 정보들이 테이블에 저장되어야 합니다.

따라서 initialize-schma를 초기화해 주고, 스키마는 postgresql으로 주었습니다.

실제 schema의 classpath로 이동해 보면 어떤 table들이 create 되는지 SQL로 확인해 볼 수 있습니다.

 

 

Docker 준비

version: '3.8'
services:
  postgresql:
    image: postgres:14.6
    # volumes:
    #   - ~/volumes/jhipster/jhip_api_first/postgresql/:/var/lib/postgresql/data/
    environment:
      - POSTGRES_DB=your_database_name
      - POSTGRES_USER=your_postgres_username
      - POSTGRES_PASSWORD=your_postgres_password
      - POSTGRES_HOST_AUTH_METHOD=trust
    # If you want to expose these ports outside your dev PC,
    # remove the "127.0.0.1:" prefix
    ports:
      - 127.0.0.1:5432:5432
    command: [ "postgres", "-c", "max_connections=1000", "-c", "max_prepared_transactions=100" ]

postgreSQL DB 구동을 위해 docker-compose 파일을 작성해 두었습니다.

 

Batch Application 구현

import com.example.study.log.logger
import org.springframework.batch.core.Job
import org.springframework.batch.core.Step
import org.springframework.batch.core.StepContribution
import org.springframework.batch.core.job.builder.JobBuilder
import org.springframework.batch.core.launch.JobLauncher
import org.springframework.batch.core.launch.support.RunIdIncrementer
import org.springframework.batch.core.repository.JobRepository
import org.springframework.batch.core.scope.context.ChunkContext
import org.springframework.batch.core.step.builder.StepBuilder
import org.springframework.batch.item.Chunk
import org.springframework.batch.item.ItemReader
import org.springframework.batch.item.ItemWriter
import org.springframework.batch.item.support.ListItemReader
import org.springframework.batch.repeat.RepeatStatus
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import org.springframework.transaction.PlatformTransactionManager


@Configuration
class BatchConfig @Autowired constructor(
    private val jobRepository: JobRepository,
    private val batchTransactionManager: PlatformTransactionManager
) {
    @Bean
    fun firstJob(): Job {
        return JobBuilder("first job", jobRepository)
            .incrementer(RunIdIncrementer())
            .start(chunkStep())
            .next(taskletStep())
            .build()
    }

    @Bean
    fun taskletStep(): Step {
        logger.info("TaskletStep is start")
        return StepBuilder("first step", jobRepository)
            .tasklet({ _: StepContribution, chunkContext: ChunkContext ->
                logger.info("This is first tasklet step")
                logger.info("SEC = {}", chunkContext.stepContext.stepExecutionContext)
                RepeatStatus.FINISHED
            }, batchTransactionManager).build()
    }

    @Bean
    fun chunkStep(): Step {
        return StepBuilder("first step", jobRepository)
            .chunk<String, String>(BATCH_SIZE, batchTransactionManager)
            .reader(beanReader())
            .writer(ClassItemWriter())
            .build()
    }

    @Bean
    fun beanReader(): ItemReader<String> {
        val data: List<String> = mutableListOf(
            "Byte",
            "Code",
            "Data",
            "Disk",
            "File",
            "Input",
            "Loop",
            "Logic",
            "Mode",
            "Node",
            "Port",
            "Query",
            "Ratio",
            "Root",
            "Route",
            "Scope",
            "Syntax",
            "Token",
            "Trace"
        )
        logger.info ("Reading item: {}", data)
        return ListItemReader(data)
    }

    class ClassItemWriter: ItemWriter<String> {
        override fun write(itmes: Chunk<out String>) {
            logger.info("Writing item start")
            itmes.forEach {
                logger.info("Writing item : {}", it)
            }
        }
    }

    companion object {
        private const val BATCH_SIZE = 5
    }
}

JobRepository, PlatformTransactionManager을 빈으로 주입받고 Job을 하나 생성 및 Step을 2개 생성합니다.

1개의 Setp은 Tasklet,  1개의 Step은 reader, writer로 구성되었습니다. (processor는 생략)

BATCH_SIZE는 5로 주어서 Chunk단위를 5로 주었습니다.

 

JobRepository는 배치작업과 관련된 메타 데이터 정보를 저장하고 관리하는 역할을 수행합니다.

PlatformTransactionManager는 트랜잭션 관리를 담당하며 데이터베이스 리소스의 트랜잭션 커밋 및 롤백을 하는 데 사용됩니다.

 

Step은 reader와 writer로 구성되었습니다.

beanReader는 @Bean을 활용하여 구성하였고, ClassItemWriter의 경우에는 ItemWriter를 상속하여 클래스를 구현하였습니다.

 

BATCH_SIZE는 5로 chunk 단위가 5개로 구분됩니다.

예를 들어 ClassItemWriter의 경우에는 Writing item start라는 로그가 남고 이후에 5개의 아이팀 로그가 남을 것입니다.

 

SpringBatchApplication 실행시키기

2023-09-18T20:43:25.983+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item start
2023-09-18T20:43:25.983+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item : Byte
2023-09-18T20:43:25.983+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item : Code
2023-09-18T20:43:25.983+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item : Data
2023-09-18T20:43:25.983+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item : Disk
2023-09-18T20:43:25.983+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item : File
2023-09-18T20:43:25.987+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item start
2023-09-18T20:43:25.987+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item : Input
2023-09-18T20:43:25.987+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item : Loop
2023-09-18T20:43:25.987+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item : Logic
2023-09-18T20:43:25.987+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item : Mode
2023-09-18T20:43:25.987+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item : Node
2023-09-18T20:43:25.989+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item start
2023-09-18T20:43:25.989+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item : Port
2023-09-18T20:43:25.989+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item : Query
2023-09-18T20:43:25.989+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item : Ratio
2023-09-18T20:43:25.989+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item : Root
2023-09-18T20:43:25.989+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item : Route
2023-09-18T20:43:25.992+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item start
2023-09-18T20:43:25.992+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item : Scope
2023-09-18T20:43:25.992+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item : Syntax
2023-09-18T20:43:25.992+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item : Token
2023-09-18T20:43:25.992+09:00  INFO 97044 --- [           main] com.example.study.log.Logging            : Writing item : Trace

Writing Item에 대한 로그가 남는 모습을 확인할 수 있습니다.

 

참고자료

https://github.com/spring-projects/spring-boot/wiki/Spring-Boot-3.0-Migration-Guide

Spring Boot 3.0 Migration Guide

https://manib.hashnode.dev/spring-batch-with-spring-boot-3

Spring Batch with Spring Boot 3.0