AWS
[AWS] MediaConvert createJob Kotlin SDK 적용
Junuuu
2022. 12. 24. 00:01
반응형
[1] [AWS] MediaConvert란? + 튜토리얼
[2] [AWS] AWS MediaConvert createJob Kotlin SDK 적용
[3] [AWS] AWS MediaConvert Jobtemplate Kotlin SDK 적용
개요
위의 예제에서 AWS Console을 통해 MediaConvert의 job을 생성하는 작업을 수행하였습니다.
이번에는 Spring, Kotlin과 AWS MediaConvert SDK를 활용하여 job을 생성해보도록 하겠습니다.
AWS Kotlin MediaConvert SDK 문서를 기반으로 작성해보겠습니다
mediaconvert build.gradle.kts 링크
Dependency 추가
dependencies {
implementation("aws.sdk.kotlin:mediaconvert:0.17.1-beta")
testImplementation("org.junit.jupiter:junit-jupiter:5.8.2")
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-core:1.6.2")
}문서에서는 3개의 의존성을 추가하고 있습니다.
또한 CreateJob.kt 문서에는 suspend를 활용하기 때문에 coroutines의존성을 추가하였습니다.
여기서 mediaconvert의 경우에는 0.17.1-beta 버전입니다.
https://mvnrepository.com/artifact/aws.sdk.kotlin/mediaconvert
메이븐 저장소에서 aws.sdk.kotlin:mediaconvert의 버전들을 살펴보았습니다.
[그림 1]을 통해 aws.sdk.kotlin:mediaconvert의 다양한 버전들을 볼 수 있습니다.
거의 다 beta가 붙어있고 0.16.0의 경우에는 붙어있지 않습니다.
따라서 0.16.0 버전을 활용하고자 합니다.
CreateJob.kt
코드가 이미 작성되어 있고 suspend fun으로 함수 구성이 되어있습니다.
코루틴은 기본적으로 일시 중단이 가능합니다.
[그림 2]는 코루틴이 일시 중단 가능한 예시를 보여줍니다.
코루틴이 일시 중단이 되려면 수행되는 위치 또한 코루틴 내부여야 하기 때문에 일시 중단 작업을 코루틴 내부로 옮기거나, suspend fun을 활용해야 합니다.
따라서 Controller 코드에 suspend가 붙게 됩니다.
@PostMapping("/job")
suspend fun createJob(){
return jobService.createMediaConvertJob()
}URL으로 /job 이 호출되면 job을 만들도록 구성하였습니다.
suspend를 제거하려고 해도 내부적으로 코루틴을 사용하기 때문에 불가능합니다.
과연 Controller에 suspend를 사용해도 될까?
코루틴은 non-blocking 으로 동작하는 Spring WebFlux에서 지원됩니다.
Spring WebFlux에서 컨트롤러 메소드는 suspend로 선언될 수 있습니다.
example WebFlux with coroutines code
@GetMapping("/foo")
suspend fun foo: X = coroutineScope{
val x = async { restClient.getForObject<Int>("/bla1") }
val y = async { restClient.getForObject<Int>("/bla2") }
x.await() + y.await()
}
Spring Boot에서는 컨트롤러를 suspending으로 선언할 수 없지만 runBlocking을 사용할 수 있습니다.
example Spring Boot with coroutines codre
@GetMapping("/foo")
fun foo(): Int = runBlocking {
val x = async { restClient.getForObject<Int>("/bla1") }
val y = async { restClient.getForObject<Int>("/bla2") }
x.await() + y.await()
}Spring Boot는 요청당 하나의 스레드 모델을 구현하므로 스레드는 어쨋든 차단됩니다.
그래서 기존의 코드를 어떻게 처리하였는지는 아래에서 다루겠습니다
다음은 Service 코드입니다.
@Service
class JobService {
suspend fun createMediaConvertJob(){
val usage = """
Usage
<mcRoleARN> <fileInput>
Where:
mcRoleARN - the MediaConvert Role ARN.
fileInput - the URL of an Amazon S3 bucket where the input file is located.
"""
val mcRoleARN = "미디어 컨버터의 iam role을 넣어주세요(arn:aws:iam::xxxxxxxxxx:role/MediaConvert-common)"
val fileInput = "mediaconvert에서 사용될 s3 input file을 넣어주세요(s3://your_bucket/your_path/somefile.mov)"
val mcClient = MediaConvertClient { region = "aws 리전을 넣어주세요(ap-northeast-2)"}
val id = createMediaJob(mcClient, mcRoleARN, fileInput)
println("MediaConvert job is $id")
}
}
suspend fun createMediaJob(mcClient: MediaConvertClient, mcRoleARN: String, fileInputVal: String): String? {
val s3path = fileInputVal.substring(0, fileInputVal.lastIndexOf('/') + 1) + "kotlin-sdk/out/"
val fileOutput = s3path + "index"
val thumbsOutput = s3path + "thumbs/"
val mp4Output = s3path + "mp4/"
try {
val describeEndpoints = DescribeEndpointsRequest {
maxResults = 20
}
val res = mcClient.describeEndpoints(describeEndpoints)
if (res.endpoints?.size!! <= 0) {
println("Cannot find MediaConvert service endpoint URL!")
exitProcess(0)
}
val endpointURL = res.endpoints!![0].url!!
val mediaConvertClient = MediaConvertClient {
region = "ap-northeast-2"
endpointResolver = AwsEndpointResolver { service, region ->
AwsEndpoint(endpointURL, CredentialScope(region = region))
}
}
// output group Preset HLS low profile
val hlsLow = createOutput("hls_low", "_low", "_\$dt$", 750000, 7, 1920, 1080, 640)
// output group Preset HLS medium profile
val hlsMedium = createOutput("hls_medium", "_medium", "_\$dt$", 1200000, 7, 1920, 1080, 1280)
// output group Preset HLS high profole
val hlsHigh = createOutput("hls_high", "_high", "_\$dt$", 3500000, 8, 1920, 1080, 1920)
val outputSettings = OutputGroupSettings {
type = OutputGroupType.HlsGroupSettings
}
val OutputObsList: MutableList<Output> = mutableListOf()
if (hlsLow != null) {
OutputObsList.add(hlsLow)
}
if (hlsMedium != null) {
OutputObsList.add(hlsMedium)
}
if (hlsHigh != null) {
OutputObsList.add(hlsHigh)
}
// Create an OutputGroup object.
val appleHLS = OutputGroup {
name = "Apple HLS"
customName = "Example"
outputGroupSettings = OutputGroupSettings {
type = OutputGroupType.HlsGroupSettings
this.hlsGroupSettings = HlsGroupSettings {
directoryStructure = HlsDirectoryStructure.SingleDirectory
manifestDurationFormat = HlsManifestDurationFormat.Integer
streamInfResolution = HlsStreamInfResolution.Include
clientCache = HlsClientCache.Enabled
captionLanguageSetting = HlsCaptionLanguageSetting.Omit
manifestCompression = HlsManifestCompression.None
codecSpecification = HlsCodecSpecification.Rfc4281
outputSelection = HlsOutputSelection.ManifestsAndSegments
programDateTime = HlsProgramDateTime.Exclude
programDateTimePeriod = 600
timedMetadataId3Frame = HlsTimedMetadataId3Frame.Priv
timedMetadataId3Period = 10
destination = fileOutput
segmentControl = HlsSegmentControl.SegmentedFiles
minFinalSegmentLength = 0.toDouble()
segmentLength = 4
minSegmentLength = 1
}
}
outputs = OutputObsList
}
val theOutput = Output {
extension = "mp4"
containerSettings = ContainerSettings {
container = ContainerType.fromValue("MP4")
}
videoDescription = VideoDescription {
width = 1280
height = 720
scalingBehavior = ScalingBehavior.Default
sharpness = 50
antiAlias = AntiAlias.Enabled
timecodeInsertion = VideoTimecodeInsertion.Disabled
colorMetadata = ColorMetadata.Insert
respondToAfd = RespondToAfd.None
afdSignaling = AfdSignaling.None
dropFrameTimecode = DropFrameTimecode.Enabled
codecSettings = VideoCodecSettings {
codec = VideoCodec.H264
h264Settings = H264Settings {
rateControlMode = H264RateControlMode.Qvbr
parControl = H264ParControl.InitializeFromSource
qualityTuningLevel = H264QualityTuningLevel.SinglePass
qvbrSettings = H264QvbrSettings { qvbrQualityLevel = 8 }
codecLevel = H264CodecLevel.Auto
codecProfile = H264CodecProfile.Main
maxBitrate = 2400000
framerateControl = H264FramerateControl.InitializeFromSource
gopSize = 2.0
gopSizeUnits = H264GopSizeUnits.Seconds
numberBFramesBetweenReferenceFrames = 2
gopClosedCadence = 1
gopBReference = H264GopBReference.Disabled
slowPal = H264SlowPal.Disabled
syntax = H264Syntax.Default
numberReferenceFrames = 3
dynamicSubGop = H264DynamicSubGop.Static
fieldEncoding = H264FieldEncoding.Paff
sceneChangeDetect = H264SceneChangeDetect.Enabled
minIInterval = 0
telecine = H264Telecine.None
framerateConversionAlgorithm = H264FramerateConversionAlgorithm.DuplicateDrop
entropyEncoding = H264EntropyEncoding.Cabac
slices = 1
unregisteredSeiTimecode = H264UnregisteredSeiTimecode.Disabled
repeatPps = H264RepeatPps.Disabled
adaptiveQuantization = H264AdaptiveQuantization.High
spatialAdaptiveQuantization = H264SpatialAdaptiveQuantization.Enabled
temporalAdaptiveQuantization = H264TemporalAdaptiveQuantization.Enabled
flickerAdaptiveQuantization = H264FlickerAdaptiveQuantization.Disabled
softness = 0
interlaceMode = H264InterlaceMode.Progressive
}
}
}
audioDescriptions = listOf(
AudioDescription {
audioTypeControl = AudioTypeControl.FollowInput
languageCodeControl = AudioLanguageCodeControl.FollowInput
codecSettings = AudioCodecSettings {
codec = AudioCodec.Aac
aacSettings = AacSettings {
codecProfile = AacCodecProfile.Lc
rateControlMode = AacRateControlMode.Cbr
codingMode = AacCodingMode.CodingMode2_0
sampleRate = 44100
bitrate = 160000
rawFormat = AacRawFormat.None
specification = AacSpecification.Mpeg4
audioDescriptionBroadcasterMix = AacAudioDescriptionBroadcasterMix.Normal
}
}
}
)
}
// Create an OutputGroup
val fileMp4 = OutputGroup {
name = "File Group"
customName = "mp4"
outputGroupSettings = OutputGroupSettings {
type = OutputGroupType.FileGroupSettings
fileGroupSettings = FileGroupSettings {
destination = mp4Output
}
}
outputs = listOf(theOutput)
}
val containerSettings1 = ContainerSettings {
container = ContainerType.Raw
}
val thumbs = OutputGroup {
name = "File Group"
customName = "thumbs"
outputGroupSettings = OutputGroupSettings {
type = OutputGroupType.FileGroupSettings
fileGroupSettings = FileGroupSettings {
destination = thumbsOutput
}
}
outputs = listOf(
Output {
extension = "jpg"
this.containerSettings = containerSettings1
videoDescription = VideoDescription {
scalingBehavior = ScalingBehavior.Default
sharpness = 50
antiAlias = AntiAlias.Enabled
timecodeInsertion = VideoTimecodeInsertion.Disabled
colorMetadata = ColorMetadata.Insert
dropFrameTimecode = DropFrameTimecode.Enabled
codecSettings = VideoCodecSettings {
codec = VideoCodec.FrameCapture
frameCaptureSettings = FrameCaptureSettings {
framerateNumerator = 1
framerateDenominator = 1
maxCaptures = 10000000
quality = 80
}
}
}
}
)
}
val audioSelectors1: MutableMap<String, AudioSelector> = HashMap()
audioSelectors1["Audio Selector 1"] =
AudioSelector {
defaultSelection = AudioDefaultSelection.Default
offset = 0
}
val jobSettings = JobSettings {
inputs = listOf(
Input {
audioSelectors = audioSelectors1
videoSelector = VideoSelector {
colorSpace = ColorSpace.Follow
rotate = InputRotate.Degree0
}
filterEnable = InputFilterEnable.Auto
filterStrength = 0
deblockFilter = InputDeblockFilter.Disabled
denoiseFilter = InputDenoiseFilter.Disabled
psiControl = InputPsiControl.UsePsi
timecodeSource = InputTimecodeSource.Embedded
fileInput = fileInputVal
outputGroups = listOf(appleHLS, thumbs, fileMp4)
}
)
}
val createJobRequest = CreateJobRequest {
role = mcRoleARN
settings = jobSettings
}
val createJobResponse = mediaConvertClient.createJob(createJobRequest)
return createJobResponse.job?.id
} catch (ex: MediaConvertException) {
println(ex.message)
mcClient.close()
exitProcess(0)
}
}
fun createOutput(
customName: String,
nameModifierVal: String,
segmentModifierVal: String,
qvbrMaxBitrate: Int,
qvbrQualityLevelVal: Int,
originWidth: Int,
originHeight: Int,
targetWidth: Int
): Output? {
val targetHeight = (
Math.round((originHeight * targetWidth / originWidth).toFloat()) -
Math.round((originHeight * targetWidth / originWidth).toFloat()) % 4
)
var output: Output? = null
try {
val audio1 = AudioDescription {
audioTypeControl = AudioTypeControl.FollowInput
languageCodeControl = AudioLanguageCodeControl.FollowInput
codecSettings = AudioCodecSettings {
codec = AudioCodec.Aac
aacSettings = AacSettings {
codecProfile = AacCodecProfile.Lc
rateControlMode = AacRateControlMode.Cbr
codingMode = AacCodingMode.CodingMode2_0
sampleRate = 44100
bitrate = 96000
rawFormat = AacRawFormat.None
specification = AacSpecification.Mpeg4
audioDescriptionBroadcasterMix = AacAudioDescriptionBroadcasterMix.Normal
}
}
}
output = Output {
nameModifier = nameModifierVal
outputSettings = OutputSettings {
hlsSettings = HlsSettings {
segmentModifier = segmentModifierVal
audioGroupId = "program_audio"
iFrameOnlyManifest = HlsIFrameOnlyManifest.Exclude
}
}
containerSettings = ContainerSettings {
container = ContainerType.M3U8
this.m3U8Settings = M3U8Settings {
audioFramesPerPes = 4
pcrControl = M3U8PcrControl.PcrEveryPesPacket
pmtPid = 480
privateMetadataPid = 503
programNumber = 1
patInterval = 0
pmtInterval = 0
scte35Source = M3U8Scte35Source.None
scte35Pid = 500
nielsenId3 = M3U8NielsenId3.None
timedMetadata = TimedMetadata.None
timedMetadataPid = 502
videoPid = 481
audioPids = listOf(482, 483, 484, 485, 486, 487, 488, 489, 490, 491, 492)
}
videoDescription = VideoDescription {
width = targetWidth
height = targetHeight
scalingBehavior = ScalingBehavior.Default
sharpness = 50
antiAlias = AntiAlias.Enabled
timecodeInsertion = VideoTimecodeInsertion.Disabled
colorMetadata = ColorMetadata.Insert
respondToAfd = RespondToAfd.None
afdSignaling = AfdSignaling.None
dropFrameTimecode = DropFrameTimecode.Enabled
codecSettings = VideoCodecSettings {
codec = VideoCodec.H264
h264Settings = H264Settings {
rateControlMode = H264RateControlMode.Qvbr
parControl = H264ParControl.InitializeFromSource
qualityTuningLevel = H264QualityTuningLevel.SinglePass
qvbrSettings = H264QvbrSettings {
qvbrQualityLevel = qvbrQualityLevelVal
}
codecLevel = H264CodecLevel.Auto
codecProfile =
if (targetHeight > 720 && targetWidth > 1280) H264CodecProfile.High else H264CodecProfile.Main
maxBitrate = qvbrMaxBitrate
framerateControl = H264FramerateControl.InitializeFromSource
gopSize = 2.0
gopSizeUnits = H264GopSizeUnits.Seconds
numberBFramesBetweenReferenceFrames = 2
gopClosedCadence = 1
gopBReference = H264GopBReference.Disabled
slowPal = H264SlowPal.Disabled
syntax = H264Syntax.Default
numberReferenceFrames = 3
dynamicSubGop = H264DynamicSubGop.Static
fieldEncoding = H264FieldEncoding.Paff
sceneChangeDetect = H264SceneChangeDetect.Enabled
minIInterval = 0
telecine = H264Telecine.None
framerateConversionAlgorithm = H264FramerateConversionAlgorithm.DuplicateDrop
entropyEncoding = H264EntropyEncoding.Cabac
slices = 1
unregisteredSeiTimecode = H264UnregisteredSeiTimecode.Disabled
repeatPps = H264RepeatPps.Disabled
adaptiveQuantization = H264AdaptiveQuantization.High
spatialAdaptiveQuantization = H264SpatialAdaptiveQuantization.Enabled
temporalAdaptiveQuantization = H264TemporalAdaptiveQuantization.Enabled
flickerAdaptiveQuantization = H264FlickerAdaptiveQuantization.Disabled
softness = 0
interlaceMode = H264InterlaceMode.Progressive
}
}
audioDescriptions = listOf(audio1)
}
}
}
} catch (ex: MediaConvertException) {
println(ex.toString())
exitProcess(0)
}
return output
}MediaConvert에서 job을 생성하기 위한 다양한 옵션들이 들어가기 때문에 코드가 좀 깁니다.
수정해야 할 부분은 다음과 같습니다.
- mcRoleARN
- fileInput
- region
- suspend 제거하기
val mcRoleARN = "미디어 컨버터의 iam role을 넣어주세요(arn:aws:iam::xxxxxxxxxx:role/MediaConvert-common)"
val fileInput = "mediaconvert에서 사용될 s3 input file을 넣어주세요(s3://your_bucket/your_path/somefile.mov)"
val mcClient = MediaConvertClient { region = "aws 리전을 넣어주세요(ap-northeast-2)"}
suspend 제거하기
Spring Boot에서는 컨트롤러를 suspending으로 선언하지 않고 runBlocking을 사용해야 하기 때문에 suspend를 모두 제거하고 runblokcing{}으로 코루틴함수를 감싸줍니다.
runblocking{}을 사용하면 하위스코프가 완전히 끝나야 그다음 라인 실행이 가능합니다.
val res = runBlocking { mcClient.describeEndpoints(describeEndpoints) }
...
...
...
val createJobResponse = runBlocking { mediaConvertClient.createJob(createJobRequest) }describeEndpoints 메서드와 createJob메서드를 타고들어가보면 내부적으로 suspend fun으로 구현되어 있습니다.
런타임 시 ClassNotFoundException
실행 후 다음과 같은 Exception이 발생할 수 있습니다.
java.lang.ClassNotFoundException: org.reactivestreams.Publisherjava.lang.ClassNotFoundException: kotlinx.coroutines.reactor.MonoKtimplementation("org.reactivestreams:reactive-streams:1.0.4")
implementation("org.jetbrains.kotlinx:kotlinx-coroutines-reactor:1.6.2")두 의존성을 implementation 하여 해결했습니다.
이제 성공적으로 job이 생성되는것을 확인할 수 있습니다.
참고자료
[Coroutine] 5. suspend fun의 이해
일시중단 가능한 코루틴 코루틴은 기본적으로 일시중단 가능하다. launch로 실행하든 async로 실행하든 내부에 해당 코루틴을 일시중단 해야하는 동작이 있으면 코루틴은 일시 중단된다. 예시로
kotlinworld.com
Why I am getting NoClassDefFoundError: org/reactivestreams/Publisher
Stream.java import io.reactivex.*; public class Stream { public static void main(String args[]) { Observable.just("Howdy!").subscribe(System.out::println); } } build.gradle:
stackoverflow.com