Hi, i'm on quincy trying to register a kafka notification; i can get non-authenticated kafka working fine, however, when i attempt to register a topic with credentials and a kafka cluster certificate i get an 'unknown error'. one question i have is where should the `ca-location` cert be, it doesn't look like the command line uploads it to the gateway? (but even without the attribute, it returns the same error). cheers, Yee. ❯ singularity pull docker://amazon/aws-cli ❯ alias s3="singularity exec aws-cli_latest.sif aws --endpoint-url https://s3dfrgw.example.com s3 --profile=prompt-processing" (register creds with profile prompt-processing) ❯ sns --region=default create-topic --name=processing-test --attributes='{"push-endpoint": "kafka://my-user:<redacted>@172.24.5.223:9094", "use-ssl": "true", "ca-location": "cluster.crt" }' --debug 2022-10-17 08:37:56,043 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/2.7.34 Python/3.9.11 Linux/4.18.0-372.16.1.el8_6.x86_64 docker/x86_64.amzn.2 2022-10-17 08:37:56,043 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['--endpoint-url', 'https://s3dfrgw.example.com', 'sns', '--profile=processing-test', '--region=default', 'create-topic', '--name=processing-test-test<https://s3dfrgw.example.com',%20'sns',%20'--profile=processing-test',%20'--region=default',%20'create-topic',%20'--name=processing-test-test>', '--attributes={"push-endpoint": "kafka://my-user:<redacted>@172.24.5.223:9094", "use-ssl": "true", "ca-location": "cluster.crt" }', '--debug'] 2022-10-17 08:37:56,064 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_s3 at 0x7fb5de42c4c0> 2022-10-17 08:37:56,064 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_ddb at 0x7fb5de606040> 2022-10-17 08:37:56,064 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method BasicCommand.add_command of <class 'awscli.customizations.configure.configure.ConfigureCommand'>> 2022-10-17 08:37:56,064 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function change_name at 0x7fb5de6a9310> 2022-10-17 08:37:56,065 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function change_name at 0x7fb5de62f160> 2022-10-17 08:37:56,065 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function alias_opsworks_cm at 0x7fb5de43de50> 2022-10-17 08:37:56,065 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_history_commands at 0x7fb5de5cbc10> 2022-10-17 08:37:56,065 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <bound method BasicCommand.add_command of <class 'awscli.customizations.devcommands.CLIDevCommand'>> 2022-10-17 08:37:56,065 - MainThread - botocore.hooks - DEBUG - Event building-command-table.main: calling handler <function add_waiters at 0x7fb5de43d0d0> 2022-10-17 08:37:56,065 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/v2/2.7.34/dist/awscli/data/cli.json 2022-10-17 08:37:56,067 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_types at 0x7fb5de4fadc0> 2022-10-17 08:37:56,067 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function no_sign_request at 0x7fb5de501940> 2022-10-17 08:37:56,067 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_verify_ssl at 0x7fb5de5018b0> 2022-10-17 08:37:56,067 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_cli_read_timeout at 0x7fb5de501a60> 2022-10-17 08:37:56,067 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <function resolve_cli_connect_timeout at 0x7fb5de5019d0> 2022-10-17 08:37:56,068 - MainThread - botocore.hooks - DEBUG - Event top-level-args-parsed: calling handler <built-in method update of dict object at 0x7fb5de3d8f00> 2022-10-17 08:37:56,068 - MainThread - botocore.session - DEBUG - Setting config variable for profile to 'processing-test' 2022-10-17 08:37:56,068 - MainThread - botocore.session - DEBUG - Setting config variable for region to 'default' 2022-10-17 08:37:56,068 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/2.7.34 Python/3.9.11 Linux/4.18.0-372.16.1.el8_6.x86_64 docker/x86_64.amzn.2 prompt/off 2022-10-17 08:37:56,068 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['--endpoint-url', 'https://s3dfrgw.example.com', 'sns', '--profile=processing-test', '--region=default', 'create-topic', '--name=processing-test-test<https://s3dfrgw.example.com',%20'sns',%20'--profile=processing-test',%20'--region=default',%20'create-topic',%20'--name=processing-test-test>', '--attributes={"push-endpoint": "kafka://my-user:<redacted>@172.24.5.223:9094", "use-ssl": "true", "ca-location": "cluster.crt" }', '--debug'] 2022-10-17 08:37:56,068 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_timestamp_parser at 0x7fb5de42caf0> 2022-10-17 08:37:56,068 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function register_uri_param_handler at 0x7fb5deeeaee0> 2022-10-17 08:37:56,068 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_binary_formatter at 0x7fb5de4203a0> 2022-10-17 08:37:56,068 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function no_pager_handler at 0x7fb5deeeb310> 2022-10-17 08:37:56,068 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_assume_role_provider_cache at 0x7fb5dee51d30> 2022-10-17 08:37:56,069 - MainThread - botocore.utils - DEBUG - IMDS ENDPOINT: http://169.254.169.254/ 2022-10-17 08:37:56,071 - MainThread - botocore.credentials - DEBUG - Skipping environment variable credential check because profile name was explicitly set. 2022-10-17 08:37:56,071 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function attach_history_handler at 0x7fb5de5cbaf0> 2022-10-17 08:37:56,071 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_json_file_cache at 0x7fb5de5ffe50> 2022-10-17 08:37:56,082 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/v2/2.7.34/dist/awscli/botocore/data/sns/2010-03-31/service-2.json 2022-10-17 08:37:56,089 - MainThread - botocore.hooks - DEBUG - Event building-command-table.sns: calling handler <function add_waiters at 0x7fb5de43d0d0> 2022-10-17 08:37:56,101 - MainThread - awscli.clidriver - DEBUG - OrderedDict([('name', <awscli.arguments.CLIArgument object at 0x7fb5dda98e80>), ('attributes', <awscli.arguments.CLIArgument object at 0x7fb5dda98eb0>), ('tags', <awscli.arguments.ListArgument object at 0x7fb5dda98df0>), ('data-protection-policy', <awscli.arguments.CLIArgument object at 0x7fb5dda98dc0>)]) 2022-10-17 08:37:56,102 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.sns.create-topic: calling handler <function add_streaming_output_arg at 0x7fb5de42cdc0> 2022-10-17 08:37:56,102 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.sns.create-topic: calling handler <function add_cli_input_json at 0x7fb5dee56550> 2022-10-17 08:37:56,102 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.sns.create-topic: calling handler <function add_cli_input_yaml at 0x7fb5dee56790> 2022-10-17 08:37:56,102 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.sns.create-topic: calling handler <function unify_paging_params at 0x7fb5de6065e0> 2022-10-17 08:37:56,115 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/v2/2.7.34/dist/awscli/botocore/data/sns/2010-03-31/paginators-1.json 2022-10-17 08:37:56,116 - MainThread - botocore.hooks - DEBUG - Event building-argument-table.sns.create-topic: calling handler <function add_generate_skeleton at 0x7fb5de4fa3a0> 2022-10-17 08:37:56,116 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.sns.create-topic: calling handler <bound method OverrideRequiredArgsArgument.override_required_args of <awscli.customizations.cliinput.CliInputJSONArgument object at 0x7fb5dda98f40>> 2022-10-17 08:37:56,116 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.sns.create-topic: calling handler <bound method OverrideRequiredArgsArgument.override_required_args of <awscli.customizations.cliinput.CliInputYAMLArgument object at 0x7fb5dda98fd0>> 2022-10-17 08:37:56,116 - MainThread - botocore.hooks - DEBUG - Event before-building-argument-table-parser.sns.create-topic: calling handler <bound method GenerateCliSkeletonArgument.override_required_args of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x7fb5dda98f70>> 2022-10-17 08:37:56,117 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.sns.create-topic.name: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7fb5ddba77f0> 2022-10-17 08:37:56,117 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.sns.create-topic: calling handler <awscli.argprocess.ParamShorthandParser object at 0x7fb5dee915b0> 2022-10-17 08:37:56,117 - MainThread - awscli.arguments - DEBUG - Unpacked value of 'processing-test-test' for parameter "name": 'processing-test-test' 2022-10-17 08:37:56,117 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.sns.create-topic.attributes: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7fb5ddba77f0> 2022-10-17 08:37:56,117 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.sns.create-topic: calling handler <awscli.argprocess.ParamShorthandParser object at 0x7fb5dee915b0> 2022-10-17 08:37:56,117 - MainThread - awscli.argprocess - DEBUG - Param attributes looks like JSON, not considered for param shorthand. 2022-10-17 08:37:56,117 - MainThread - awscli.arguments - DEBUG - Unpacked value of '{"push-endpoint": "kafka://my-user:<redacted>@172.24.5.223:9094", "use-ssl": "true", "ca-location": "cluster.crt" }' for parameter "attributes": OrderedDict([('push-endpoint', 'kafka://my-user:<redacted>@172.24.5.223:9094'), ('use-ssl', 'true'), ('ca-location', 'cluster.crt')]) 2022-10-17 08:37:56,117 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.sns.create-topic.tags: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7fb5ddba77f0> 2022-10-17 08:37:56,117 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.sns.create-topic.data-protection-policy: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7fb5ddba77f0> 2022-10-17 08:37:56,117 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.sns.create-topic.cli-input-json: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7fb5ddba77f0> 2022-10-17 08:37:56,117 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.sns.create-topic.cli-input-yaml: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7fb5ddba77f0> 2022-10-17 08:37:56,117 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.sns.create-topic.generate-cli-skeleton: calling handler <awscli.paramfile.URIArgumentHandler object at 0x7fb5ddba77f0> 2022-10-17 08:37:56,117 - MainThread - botocore.hooks - DEBUG - Event calling-command.sns.create-topic: calling handler <bound method CliInputArgument.add_to_call_parameters of <awscli.customizations.cliinput.CliInputJSONArgument object at 0x7fb5dda98f40>> 2022-10-17 08:37:56,117 - MainThread - botocore.hooks - DEBUG - Event calling-command.sns.create-topic: calling handler <bound method CliInputArgument.add_to_call_parameters of <awscli.customizations.cliinput.CliInputYAMLArgument object at 0x7fb5dda98fd0>> 2022-10-17 08:37:56,117 - MainThread - botocore.hooks - DEBUG - Event calling-command.sns.create-topic: calling handler <bound method GenerateCliSkeletonArgument.generate_skeleton of <awscli.customizations.generatecliskeleton.GenerateCliSkeletonArgument object at 0x7fb5dda98f70>> 2022-10-17 08:37:56,118 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role 2022-10-17 08:37:56,118 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role-with-web-identity 2022-10-17 08:37:56,118 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: sso 2022-10-17 08:37:56,118 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file 2022-10-17 08:37:56,118 - MainThread - botocore.credentials - INFO - Found credentials in shared credentials file: ~/.aws/credentials 2022-10-17 08:37:56,120 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/aws-cli/v2/2.7.34/dist/awscli/botocore/data/endpoints.json 2022-10-17 08:37:56,133 - MainThread - botocore.hooks - DEBUG - Event choose-service-name: calling handler <function handle_service_name_alias at 0x7fb5e0a6a5e0> 2022-10-17 08:37:56,135 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.sns: calling handler <function add_generate_presigned_url at 0x7fb5e0ad5310> 2022-10-17 08:37:56,135 - MainThread - botocore.regions - DEBUG - Creating a regex based endpoint for sns, default 2022-10-17 08:37:56,138 - MainThread - botocore.endpoint - DEBUG - Setting sns timeout as (60, 60) 2022-10-17 08:37:56,139 - MainThread - botocore.hooks - DEBUG - Event provide-client-params.sns.CreateTopic: calling handler <function base64_decode_input_blobs at 0x7fb5de420af0> 2022-10-17 08:37:56,139 - MainThread - botocore.hooks - DEBUG - Event before-parameter-build.sns.CreateTopic: calling handler <function generate_idempotent_uuid at 0x7fb5e0801700> 2022-10-17 08:37:56,139 - MainThread - botocore.hooks - DEBUG - Event before-call.sns.CreateTopic: calling handler <function inject_api_version_header_if_needed at 0x7fb5e0802f70> 2022-10-17 08:37:56,139 - MainThread - botocore.endpoint - DEBUG - Making request for OperationModel(name=CreateTopic) with params: {'url_path': '/', 'query_string': '', 'method': 'POST', 'headers': {'Content-Type': 'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': 'aws-cli/2.7.34 Python/3.9.11 Linux/4.18.0-372.16.1.el8_6.x86_64 docker/x86_64.amzn.2 prompt/off command/sns.create-topic'}, 'body': {'Action': 'CreateTopic', 'Version': '2010-03-31', 'Name': 'processing-test-test', 'Attributes.entry.1.key': 'push-endpoint', 'Attributes.entry.1.value': 'kafka://my-user:<redacted>@172.24.5.223:9094', 'Attributes.entry.2.key': 'use-ssl', 'Attributes.entry.2.value': 'true', 'Attributes.entry.3.key': 'ca-location', 'Attributes.entry.3.value': 'cluster.crt'}, 'url': 'https://s3dfrgw.example.com/', 'context<https://s3dfrgw.example.com/',%20'context>': {'client_region': 'default', 'client_config': <botocore.config.Config object at 0x7fb5dd567b20>, 'has_streaming_input': False, 'auth_type': None}} 2022-10-17 08:37:56,139 - MainThread - botocore.hooks - DEBUG - Event request-created.sns.CreateTopic: calling handler <bound method RequestSigner.handler of <botocore.signers.RequestSigner object at 0x7fb5dd5679d0>> 2022-10-17 08:37:56,139 - MainThread - botocore.hooks - DEBUG - Event choose-signer.sns.CreateTopic: calling handler <function set_operation_specific_signer at 0x7fb5e08015e0> 2022-10-17 08:37:56,139 - MainThread - botocore.auth - DEBUG - Calculating signature using v4 auth. 2022-10-17 08:37:56,140 - MainThread - botocore.auth - DEBUG - CanonicalRequest: POST / content-type:application/x-www-form-urlencoded; charset=utf-8 host:s3dfrgw.example.com<http://s3dfrgw.example.com> x-amz-date:20221017T153756Z content-type;host;x-amz-date 9cd4cabc69d03c0b45b51de3aed2764185b2d3e4a2c426d95624ee04127d2c89 2022-10-17 08:37:56,140 - MainThread - botocore.auth - DEBUG - StringToSign: AWS4-HMAC-SHA256 20221017T153756Z 20221017/default/sns/aws4_request 6118de5b5cc8466768a15393fba5f8c7ffa8ee212e2f3555086774d69b2493bc 2022-10-17 08:37:56,140 - MainThread - botocore.auth - DEBUG - Signature: 62d6119ffa70fcbffaa476dfc400402e174e6734e2d3174172c66c56e00b4a75 2022-10-17 08:37:56,140 - MainThread - botocore.endpoint - DEBUG - Sending http request: <AWSPreparedRequest stream_output=False, method=POST, url=https://s3dfrgw.example.com/, headers={'Content-Type': b'application/x-www-form-urlencoded; charset=utf-8', 'User-Agent': b'aws-cli/2.7.34 Python/3.9.11 Linux/4.18.0-372.16.1.el8_6.x86_64 docker/x86_64.amzn.2 prompt/off command/sns.create-topic', 'X-Amz-Date': b'20221017T153756Z', 'Authorization': b'AWS4-HMAC-SHA256 Credential=8PTGE4UNTO2ELRK3EFOR/20221017/default/sns/aws4_request, SignedHeaders=content-type;host;x-amz-date, Signature=62d6119ffa70fcbffaa476dfc400402e174e6734e2d3174172c66c56e00b4a75', 'Content-Length': '326'}> 2022-10-17 08:37:56,140 - MainThread - botocore.httpsession - DEBUG - Certificate path: /usr/local/aws-cli/v2/2.7.34/dist/awscli/botocore/cacert.pem 2022-10-17 08:37:56,140 - MainThread - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): s3dfrgw.example.com:443<http://s3dfrgw.example.com:443> 2022-10-17 08:37:56,167 - MainThread - urllib3.connectionpool - DEBUG - https://s3dfrgw.example.com:443 "POST / HTTP/1.1" 400 187 2022-10-17 08:37:56,167 - MainThread - botocore.parsers - DEBUG - Response headers: {'Content-Length': '187', 'x-amz-request-id': 'tx000005ec21219335dafaa-00634d76d4-2f74a-default', 'Accept-Ranges': 'bytes', 'Content-Type': 'application/xml', 'Date': 'Mon, 17 Oct 2022 15:37:56 GMT'} 2022-10-17 08:37:56,167 - MainThread - botocore.parsers - DEBUG - Response body: b'<?xml version="1.0" encoding="UTF-8"?><Error><Code>InvalidInput</Code><RequestId>tx000005ec21219335dafaa-00634d76d4-2f74a-default</RequestId><HostId>2f74a-default-default</HostId></Error>' 2022-10-17 08:37:56,168 - MainThread - botocore.hooks - DEBUG - Event needs-retry.sns.CreateTopic: calling handler <bound method RetryHandler.needs_retry of <botocore.retries.standard.RetryHandler object at 0x7fb5dd99e400>> 2022-10-17 08:37:56,168 - MainThread - botocore.retries.standard - DEBUG - Not retrying request. 2022-10-17 08:37:56,168 - MainThread - botocore.hooks - DEBUG - Event after-call.sns.CreateTopic: calling handler <bound method RetryQuotaChecker.release_retry_quota of <botocore.retries.standard.RetryQuotaChecker object at 0x7fb5dd567f70>> 2022-10-17 08:37:56,169 - MainThread - awscli.clidriver - DEBUG - Exception caught in main() Traceback (most recent call last): File "awscli/clidriver.py", line 458, in main File "awscli/clidriver.py", line 593, in __call__ File "awscli/clidriver.py", line 769, in __call__ File "awscli/clidriver.py", line 900, in invoke File "awscli/clidriver.py", line 912, in _make_client_call File "awscli/botocore/client.py", line 309, in _api_call File "awscli/botocore/client.py", line 639, in _make_api_call botocore.exceptions.ClientError: An error occurred (Unknown) when calling the CreateTopic operation: Unknown An error occurred (Unknown) when calling the CreateTopic operation: Unknown _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |