[AWS] MediaConvert createJob Kotlin SDK 적용

[1] [AWS] MediaConvert란? + 튜토리얼

[2] [AWS] AWS MediaConvert createJob Kotlin SDK 적용

[3] [AWS] AWS MediaConvert Jobtemplate Kotlin SDK 적용

개요

위의 예제에서 AWS Console을 통해 MediaConvert의 job을 생성하는 작업을 수행하였습니다.

이번에는 Spring, Kotlin과 AWS MediaConvert SDK를 활용하여 job을 생성해보도록 하겠습니다.

 

AWS Kotlin MediaConvert SDK 문서를 기반으로 작성해보겠습니다

mediaconvert build.gradle.kts 링크

mediaconvert CreateJob.kt 링크

 

 

Dependency 추가

dependencies {
    implementation("aws.sdk.kotlin:mediaconvert:0.17.1-beta")
    testImplementation("org.junit.jupiter:junit-jupiter:5.8.2")
    implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.2")
}

문서에서는 3개의 의존성을 추가하고 있습니다.

 

또한 CreateJob.kt 문서에는 suspend를 활용하기 때문에 coroutines의존성을 추가하였습니다.

 

여기서 mediaconvert의 경우에는 0.17.1-beta 버전입니다.

 

https://mvnrepository.com/artifact/aws.sdk.kotlin/mediaconvert

 

메이븐 저장소에서 aws.sdk.kotlin:mediaconvert의 버전들을 살펴보았습니다.

그림 1

[그림 1]을 통해 aws.sdk.kotlin:mediaconvert의 다양한 버전들을 볼 수 있습니다.

거의 다 beta가 붙어있고 0.16.0의 경우에는 붙어있지 않습니다.

따라서 0.16.0 버전을 활용하고자 합니다.

 

CreateJob.kt

코드가 이미 작성되어 있고 suspend fun으로 함수 구성이 되어있습니다.

 

코루틴은 기본적으로 일시 중단이 가능합니다.

 

[그림 2]는 코루틴이 일시 중단 가능한 예시를 보여줍니다.

그림 2

 

코루틴이 일시 중단이 되려면 수행되는 위치 또한 코루틴 내부여야 하기 때문에 일시 중단 작업을 코루틴 내부로 옮기거나, suspend fun을 활용해야 합니다.

 

따라서 Controller 코드에 suspend가 붙게 됩니다.

@PostMapping("/job")
suspend fun createJob(){
	return jobService.createMediaConvertJob()
}

URL으로 /job 이 호출되면 job을 만들도록 구성하였습니다.

suspend를 제거하려고 해도 내부적으로 코루틴을 사용하기 때문에 불가능합니다.

 

과연 Controller에 suspend를 사용해도 될까?

코루틴은 non-blocking 으로 동작하는 Spring WebFlux에서 지원됩니다.

Spring WebFlux에서 컨트롤러 메소드는 suspend로 선언될 수 있습니다.

 

example WebFlux with coroutines code 

@GetMapping("/foo")
suspend fun foo: X = coroutineScope{
    val x = async { restClient.getForObject<Int>("/bla1") }
    val y = async { restClient.getForObject<Int>("/bla2") }
    x.await() + y.await()
}

 

Spring Boot에서는 컨트롤러를 suspending으로 선언할 수 없지만 runBlocking을 사용할 수 있습니다.

 

example Spring Boot with coroutines codre

@GetMapping("/foo")
fun foo(): Int = runBlocking {
    val x = async { restClient.getForObject<Int>("/bla1") }
    val y = async { restClient.getForObject<Int>("/bla2") }
    x.await() + y.await()
}

Spring Boot는 요청당 하나의 스레드 모델을 구현하므로 스레드는 어쨋든 차단됩니다.

그래서 기존의 코드를 어떻게 처리하였는지는 아래에서 다루겠습니다

 

다음은 Service 코드입니다.

@Service
class JobService {
    suspend fun createMediaConvertJob(){
        val usage = """
        Usage
            <mcRoleARN> <fileInput> 

        Where:
            mcRoleARN - the MediaConvert Role ARN.
            fileInput -  the URL of an Amazon S3 bucket where the input file is located.
        """

        val mcRoleARN = "미디어 컨버터의 iam role을 넣어주세요(arn:aws:iam::xxxxxxxxxx:role/MediaConvert-common)"
        val fileInput = "mediaconvert에서 사용될 s3 input file을 넣어주세요(s3://your_bucket/your_path/somefile.mov)"
        val mcClient = MediaConvertClient { region = "aws 리전을 넣어주세요(ap-northeast-2)"}
        val id = createMediaJob(mcClient, mcRoleARN, fileInput)
        println("MediaConvert job is $id")
    }
}

suspend fun createMediaJob(mcClient: MediaConvertClient, mcRoleARN: String, fileInputVal: String): String? {
    val s3path = fileInputVal.substring(0, fileInputVal.lastIndexOf('/') + 1) + "kotlin-sdk/out/"
    val fileOutput = s3path + "index"
    val thumbsOutput = s3path + "thumbs/"
    val mp4Output = s3path + "mp4/"

    try {
        val describeEndpoints = DescribeEndpointsRequest {
            maxResults = 20
        }

        val res = mcClient.describeEndpoints(describeEndpoints)

        if (res.endpoints?.size!! <= 0) {
            println("Cannot find MediaConvert service endpoint URL!")
            exitProcess(0)
        }
        val endpointURL = res.endpoints!![0].url!!
        val mediaConvertClient = MediaConvertClient {
            region = "ap-northeast-2"
            endpointResolver = AwsEndpointResolver { service, region ->
                AwsEndpoint(endpointURL, CredentialScope(region = region))
            }
        }

        // output group Preset HLS low profile
        val hlsLow = createOutput("hls_low", "_low", "_\$dt$", 750000, 7, 1920, 1080, 640)

        // output group Preset HLS medium profile
        val hlsMedium = createOutput("hls_medium", "_medium", "_\$dt$", 1200000, 7, 1920, 1080, 1280)

        // output group Preset HLS high profole
        val hlsHigh = createOutput("hls_high", "_high", "_\$dt$", 3500000, 8, 1920, 1080, 1920)

        val outputSettings = OutputGroupSettings {
            type = OutputGroupType.HlsGroupSettings
        }

        val OutputObsList: MutableList<Output> = mutableListOf()
        if (hlsLow != null) {
            OutputObsList.add(hlsLow)
        }
        if (hlsMedium != null) {
            OutputObsList.add(hlsMedium)
        }
        if (hlsHigh != null) {
            OutputObsList.add(hlsHigh)
        }

        // Create an OutputGroup object.
        val appleHLS = OutputGroup {
            name = "Apple HLS"
            customName = "Example"
            outputGroupSettings = OutputGroupSettings {
                type = OutputGroupType.HlsGroupSettings
                this.hlsGroupSettings = HlsGroupSettings {
                    directoryStructure = HlsDirectoryStructure.SingleDirectory
                    manifestDurationFormat = HlsManifestDurationFormat.Integer
                    streamInfResolution = HlsStreamInfResolution.Include
                    clientCache = HlsClientCache.Enabled
                    captionLanguageSetting = HlsCaptionLanguageSetting.Omit
                    manifestCompression = HlsManifestCompression.None
                    codecSpecification = HlsCodecSpecification.Rfc4281
                    outputSelection = HlsOutputSelection.ManifestsAndSegments
                    programDateTime = HlsProgramDateTime.Exclude
                    programDateTimePeriod = 600
                    timedMetadataId3Frame = HlsTimedMetadataId3Frame.Priv
                    timedMetadataId3Period = 10
                    destination = fileOutput
                    segmentControl = HlsSegmentControl.SegmentedFiles
                    minFinalSegmentLength = 0.toDouble()
                    segmentLength = 4
                    minSegmentLength = 1
                }
            }
            outputs = OutputObsList
        }

        val theOutput = Output {
            extension = "mp4"
            containerSettings = ContainerSettings {
                container = ContainerType.fromValue("MP4")
            }

            videoDescription = VideoDescription {
                width = 1280
                height = 720
                scalingBehavior = ScalingBehavior.Default
                sharpness = 50
                antiAlias = AntiAlias.Enabled
                timecodeInsertion = VideoTimecodeInsertion.Disabled
                colorMetadata = ColorMetadata.Insert
                respondToAfd = RespondToAfd.None
                afdSignaling = AfdSignaling.None
                dropFrameTimecode = DropFrameTimecode.Enabled
                codecSettings = VideoCodecSettings {
                    codec = VideoCodec.H264
                    h264Settings = H264Settings {
                        rateControlMode = H264RateControlMode.Qvbr
                        parControl = H264ParControl.InitializeFromSource
                        qualityTuningLevel = H264QualityTuningLevel.SinglePass
                        qvbrSettings = H264QvbrSettings { qvbrQualityLevel = 8 }
                        codecLevel = H264CodecLevel.Auto
                        codecProfile = H264CodecProfile.Main
                        maxBitrate = 2400000
                        framerateControl = H264FramerateControl.InitializeFromSource
                        gopSize = 2.0
                        gopSizeUnits = H264GopSizeUnits.Seconds
                        numberBFramesBetweenReferenceFrames = 2
                        gopClosedCadence = 1
                        gopBReference = H264GopBReference.Disabled
                        slowPal = H264SlowPal.Disabled
                        syntax = H264Syntax.Default
                        numberReferenceFrames = 3
                        dynamicSubGop = H264DynamicSubGop.Static
                        fieldEncoding = H264FieldEncoding.Paff
                        sceneChangeDetect = H264SceneChangeDetect.Enabled
                        minIInterval = 0
                        telecine = H264Telecine.None
                        framerateConversionAlgorithm = H264FramerateConversionAlgorithm.DuplicateDrop
                        entropyEncoding = H264EntropyEncoding.Cabac
                        slices = 1
                        unregisteredSeiTimecode = H264UnregisteredSeiTimecode.Disabled
                        repeatPps = H264RepeatPps.Disabled
                        adaptiveQuantization = H264AdaptiveQuantization.High
                        spatialAdaptiveQuantization = H264SpatialAdaptiveQuantization.Enabled
                        temporalAdaptiveQuantization = H264TemporalAdaptiveQuantization.Enabled
                        flickerAdaptiveQuantization = H264FlickerAdaptiveQuantization.Disabled
                        softness = 0
                        interlaceMode = H264InterlaceMode.Progressive
                    }
                }
            }

            audioDescriptions = listOf(
                AudioDescription {
                    audioTypeControl = AudioTypeControl.FollowInput
                    languageCodeControl = AudioLanguageCodeControl.FollowInput
                    codecSettings = AudioCodecSettings {
                        codec = AudioCodec.Aac
                        aacSettings = AacSettings {
                            codecProfile = AacCodecProfile.Lc
                            rateControlMode = AacRateControlMode.Cbr
                            codingMode = AacCodingMode.CodingMode2_0
                            sampleRate = 44100
                            bitrate = 160000
                            rawFormat = AacRawFormat.None
                            specification = AacSpecification.Mpeg4
                            audioDescriptionBroadcasterMix = AacAudioDescriptionBroadcasterMix.Normal
                        }
                    }
                }
            )
        }

        // Create an OutputGroup
        val fileMp4 = OutputGroup {
            name = "File Group"
            customName = "mp4"
            outputGroupSettings = OutputGroupSettings {
                type = OutputGroupType.FileGroupSettings
                fileGroupSettings = FileGroupSettings {
                    destination = mp4Output
                }
            }
            outputs = listOf(theOutput)
        }

        val containerSettings1 = ContainerSettings {
            container = ContainerType.Raw
        }

        val thumbs = OutputGroup {
            name = "File Group"
            customName = "thumbs"
            outputGroupSettings = OutputGroupSettings {
                type = OutputGroupType.FileGroupSettings
                fileGroupSettings = FileGroupSettings {
                    destination = thumbsOutput
                }
            }

            outputs = listOf(
                Output {
                    extension = "jpg"

                    this.containerSettings = containerSettings1
                    videoDescription = VideoDescription {
                        scalingBehavior = ScalingBehavior.Default
                        sharpness = 50
                        antiAlias = AntiAlias.Enabled
                        timecodeInsertion = VideoTimecodeInsertion.Disabled
                        colorMetadata = ColorMetadata.Insert
                        dropFrameTimecode = DropFrameTimecode.Enabled
                        codecSettings = VideoCodecSettings {
                            codec = VideoCodec.FrameCapture
                            frameCaptureSettings = FrameCaptureSettings {
                                framerateNumerator = 1
                                framerateDenominator = 1
                                maxCaptures = 10000000
                                quality = 80
                            }
                        }
                    }
                }
            )
        }

        val audioSelectors1: MutableMap<String, AudioSelector> = HashMap()
        audioSelectors1["Audio Selector 1"] =
            AudioSelector {
                defaultSelection = AudioDefaultSelection.Default
                offset = 0
            }

        val jobSettings = JobSettings {
            inputs = listOf(
                Input {
                    audioSelectors = audioSelectors1
                    videoSelector = VideoSelector {
                        colorSpace = ColorSpace.Follow
                        rotate = InputRotate.Degree0
                    }
                    filterEnable = InputFilterEnable.Auto
                    filterStrength = 0
                    deblockFilter = InputDeblockFilter.Disabled
                    denoiseFilter = InputDenoiseFilter.Disabled
                    psiControl = InputPsiControl.UsePsi
                    timecodeSource = InputTimecodeSource.Embedded
                    fileInput = fileInputVal

                    outputGroups = listOf(appleHLS, thumbs, fileMp4)
                }
            )
        }

        val createJobRequest = CreateJobRequest {
            role = mcRoleARN
            settings = jobSettings
        }

        val createJobResponse = mediaConvertClient.createJob(createJobRequest)
        return createJobResponse.job?.id
    } catch (ex: MediaConvertException) {
        println(ex.message)
        mcClient.close()
        exitProcess(0)
    }
}

fun createOutput(
    customName: String,
    nameModifierVal: String,
    segmentModifierVal: String,
    qvbrMaxBitrate: Int,
    qvbrQualityLevelVal: Int,
    originWidth: Int,
    originHeight: Int,
    targetWidth: Int
): Output? {

    val targetHeight = (
            Math.round((originHeight * targetWidth / originWidth).toFloat()) -
                    Math.round((originHeight * targetWidth / originWidth).toFloat()) % 4
            )

    var output: Output? = null

    try {

        val audio1 = AudioDescription {
            audioTypeControl = AudioTypeControl.FollowInput
            languageCodeControl = AudioLanguageCodeControl.FollowInput
            codecSettings = AudioCodecSettings {
                codec = AudioCodec.Aac
                aacSettings = AacSettings {
                    codecProfile = AacCodecProfile.Lc
                    rateControlMode = AacRateControlMode.Cbr
                    codingMode = AacCodingMode.CodingMode2_0
                    sampleRate = 44100
                    bitrate = 96000
                    rawFormat = AacRawFormat.None
                    specification = AacSpecification.Mpeg4
                    audioDescriptionBroadcasterMix = AacAudioDescriptionBroadcasterMix.Normal
                }
            }
        }

        output = Output {
            nameModifier = nameModifierVal
            outputSettings = OutputSettings {
                hlsSettings = HlsSettings {
                    segmentModifier = segmentModifierVal
                    audioGroupId = "program_audio"
                    iFrameOnlyManifest = HlsIFrameOnlyManifest.Exclude
                }
            }
            containerSettings = ContainerSettings {
                container = ContainerType.M3U8
                this.m3U8Settings = M3U8Settings {
                    audioFramesPerPes = 4
                    pcrControl = M3U8PcrControl.PcrEveryPesPacket
                    pmtPid = 480
                    privateMetadataPid = 503
                    programNumber = 1
                    patInterval = 0
                    pmtInterval = 0
                    scte35Source = M3U8Scte35Source.None
                    scte35Pid = 500
                    nielsenId3 = M3U8NielsenId3.None
                    timedMetadata = TimedMetadata.None
                    timedMetadataPid = 502
                    videoPid = 481
                    audioPids = listOf(482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 492)
                }

                videoDescription = VideoDescription {
                    width = targetWidth
                    height = targetHeight
                    scalingBehavior = ScalingBehavior.Default
                    sharpness = 50
                    antiAlias = AntiAlias.Enabled
                    timecodeInsertion = VideoTimecodeInsertion.Disabled
                    colorMetadata = ColorMetadata.Insert
                    respondToAfd = RespondToAfd.None
                    afdSignaling = AfdSignaling.None
                    dropFrameTimecode = DropFrameTimecode.Enabled
                    codecSettings = VideoCodecSettings {
                        codec = VideoCodec.H264
                        h264Settings = H264Settings {
                            rateControlMode = H264RateControlMode.Qvbr
                            parControl = H264ParControl.InitializeFromSource
                            qualityTuningLevel = H264QualityTuningLevel.SinglePass
                            qvbrSettings = H264QvbrSettings {
                                qvbrQualityLevel = qvbrQualityLevelVal
                            }
                            codecLevel = H264CodecLevel.Auto
                            codecProfile =
                                if (targetHeight > 720 && targetWidth > 1280) H264CodecProfile.High else H264CodecProfile.Main
                            maxBitrate = qvbrMaxBitrate
                            framerateControl = H264FramerateControl.InitializeFromSource
                            gopSize = 2.0
                            gopSizeUnits = H264GopSizeUnits.Seconds
                            numberBFramesBetweenReferenceFrames = 2
                            gopClosedCadence = 1
                            gopBReference = H264GopBReference.Disabled
                            slowPal = H264SlowPal.Disabled
                            syntax = H264Syntax.Default
                            numberReferenceFrames = 3
                            dynamicSubGop = H264DynamicSubGop.Static
                            fieldEncoding = H264FieldEncoding.Paff
                            sceneChangeDetect = H264SceneChangeDetect.Enabled
                            minIInterval = 0
                            telecine = H264Telecine.None
                            framerateConversionAlgorithm = H264FramerateConversionAlgorithm.DuplicateDrop
                            entropyEncoding = H264EntropyEncoding.Cabac
                            slices = 1
                            unregisteredSeiTimecode = H264UnregisteredSeiTimecode.Disabled
                            repeatPps = H264RepeatPps.Disabled
                            adaptiveQuantization = H264AdaptiveQuantization.High
                            spatialAdaptiveQuantization = H264SpatialAdaptiveQuantization.Enabled
                            temporalAdaptiveQuantization = H264TemporalAdaptiveQuantization.Enabled
                            flickerAdaptiveQuantization = H264FlickerAdaptiveQuantization.Disabled
                            softness = 0
                            interlaceMode = H264InterlaceMode.Progressive
                        }
                    }
                    audioDescriptions = listOf(audio1)
                }
            }
        }
    } catch (ex: MediaConvertException) {
        println(ex.toString())
        exitProcess(0)
    }
    return output
}

MediaConvert에서 job을 생성하기 위한 다양한 옵션들이 들어가기 때문에 코드가 좀 깁니다.

 

수정해야 할 부분은 다음과 같습니다.

- mcRoleARN

- fileInput

- region

- suspend 제거하기

val mcRoleARN = "미디어 컨버터의 iam role을 넣어주세요(arn:aws:iam::xxxxxxxxxx:role/MediaConvert-common)"
val fileInput = "mediaconvert에서 사용될 s3 input file을 넣어주세요(s3://your_bucket/your_path/somefile.mov)"
val mcClient = MediaConvertClient { region = "aws 리전을 넣어주세요(ap-northeast-2)"}

 

suspend 제거하기

Spring Boot에서는 컨트롤러를 suspending으로 선언하지 않고 runBlocking을 사용해야 하기 때문에 suspend를 모두 제거하고 runblokcing{}으로 코루틴함수를 감싸줍니다.

 

runblocking{}을 사용하면 하위스코프가 완전히 끝나야 그다음 라인 실행이 가능합니다.

val res = runBlocking { mcClient.describeEndpoints(describeEndpoints) }
...
...
...
val createJobResponse = runBlocking { mediaConvertClient.createJob(createJobRequest) }

describeEndpoints 메서드와 createJob메서드를 타고들어가보면 내부적으로 suspend fun으로 구현되어 있습니다.

 

 

런타임 시 ClassNotFoundException

실행 후 다음과 같은 Exception이 발생할 수 있습니다.

java.lang.ClassNotFoundException: org.reactivestreams.Publisher

java.lang.ClassNotFoundException: kotlinx.coroutines.reactor.MonoKt

implementation("org.reactivestreams:reactive-streams:1.0.4")
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-reactor:1.6.2")

두 의존성을 implementation 하여 해결했습니다.

 

 

이제 성공적으로 job이 생성되는것을 확인할 수 있습니다.

 

 

 

 

 

참고자료

https://kotlinworld.com/m/144

[Coroutine] 5. suspend fun의 이해

https://stackoverflow.com/questions/43231477/why-i-am-getting-noclassdeffounderror-org-reactivestreams-publisher

Why I am getting NoClassDefFoundError: org/reactivestreams/Publisher