
Jaffer Syed (Autodesk Inc) asked a question.
I subscribed to the progress driver via AWS Marketplace and followed all the instructions as in the Youtube video, but the AWS Glue Job keep failing,
Can you please share the exact format of HostName - that is needed to be store in secrets manager
Below are some errors are from the log
An error occurred while calling o86.pyWriteDynamicFrame. null
2021-04-13 08:12:34,264 ERROR [main] glue.ProcessLauncher (Logging.scala:logError(70)): Exception in User Class: java.lang.reflect.UndeclaredThrowableException
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1862)
org.apache.spark.deploy.SparkHadoopUtil.runAsSparkUser(SparkHadoopUtil.scala:64)
org.apache.spark.executor.CoarseGrainedExecutorBackend$.run(CoarseGrainedExecutorBackend.scala:188)
org.apache.spark.executor.CoarseGrainedExecutorBackend$.main(CoarseGrainedExecutorBackend.scala:281)
org.apache.spark.executor.CoarseGrainedExecutorBackendPlugin$class.launch(CoarseGrainedExecutorBackendWrapper.scala:10)
org.apache.spark.executor.CoarseGrainedExecutorBackendWrapper$$anon$1.launch(CoarseGrainedExecutorBackendWrapper.scala:15)
org.apache.spark.executor.CoarseGrainedExecutorBackendWrapper.launch(CoarseGrainedExecutorBackendWrapper.scala:19)
org.apache.spark.executor.CoarseGrainedExecutorBackendWrapper$.main(CoarseGrainedExecutorBackendWrapper.scala:5)
org.apache.spark.executor.CoarseGrainedExecutorBackendWrapper.main(CoarseGrainedExecutorBackendWrapper.scala)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:498)
com.amazonaws.services.glue.SparkProcessLauncherPlugin$class.invoke(ProcessLauncher.scala:44)
com.amazonaws.services.glue.ProcessLauncher$$anon$1.invoke(ProcessLauncher.scala:75)
com.amazonaws.services.glue.ProcessLauncher.launch(ProcessLauncher.scala:114)
com.amazonaws.services.glue.ProcessLauncher$.main(ProcessLauncher.scala:26)
com.amazonaws.services.glue.ProcessLauncher.main(ProcessLauncher.scala)
Sorry you are having issues. I can provide you with sample host name. When testing the S4Hana connector my host name looked sort like this (I have obscured the part of it):
https://my9999999-api.s4hana.ondemand.com
I am trying the same format, but no luck, also we have the port number that is required, should I included at the end
https://myaddress.whitney.dbaas.ondemand.com
Thanks
Yes, I think adding it to the end would be the way to add it using a colon separator. Do you have other tracebacks you can share? The trace back you shared does not have the S4Hana connector involved or any errors that would typically be encountered when failing to connect. I know from my experience, making sure the IAM permissions have been configured correctly is important otherwise the errors never made much sense.
Another thing to consider is reaching out to AWS glue support. If your job is not actually reaching the S4Hana connector, they can look at your logs to make sure the job is configured correctly. I have had to do this to discover things like the IAM permissions were incorrectly configured.
Jaffer - if you haven't confirmed the permissions yet you can find documentation here - Updating IAM role permissions (progress.com)