Re: Openssl asynchronous operation in real network

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi Matt,
 
        Thanks for your update. Will try to implement one scheme using the same. 

        I have one doubt regarding the points which we have discussed earlier. 

        I am now facing an issue of getting an invalid waitctx address while trying to get it using ASYNC_get_wait_ctx(). I have checked using gdb, in which got memory inaccessible print when I tried to dump the waitctx structure. If I put some delay in the engine, things are working fine, ie. no corruption of waitctx structure address.

        But I am just copying the address of job structure before the ASYNC_pause().  After the HW operation is completed I will use this address to get waitctx and fd using ASYNC_get_wait_ctx() and ASYNC_WAIT_CTX_get_fd() from the thread which  I have created from the engine. I haven't access the job address directly from the thread.

        I know you have already told me that I should be careful while handling ASYNC_JOB structure from outside threads. But I have only a little experience on handling these ASYNC_JOB structures. If you can provide a solution for this, it would be helpful.

On 14 Jan 2019 5:33 pm, "Matt Caswell" <matt@xxxxxxxxxxx> wrote:


On 13/01/2019 04:36, Ananthu Unnikrishnan wrote:
> Hi Matt,
>
>             Thanks a lot for the response. The alternative method to resume job
> operation is a good one. 
>              
>             I need some more clarifications regarding the async job operation.
> Please help.
>             In ssl/ss_lib.c, ssl_start_async_job() function found. In which
> ASYNC_start_job function is calling but no mechanism found for polling the fd's.
> Is it written elsewhere?.

You are supposed to write the fd polling code yourself, e.g. by using
SSL_get_all_async_fds() or SSL_get_changed_async_fds() to get hold of the fds,
and then followed by "select" or similar.

See:

https://www.openssl.org/docs/man1.1.1/man3/SSL_get_all_async_fds.html

mATT



>            
>
> On Fri, Jan 11, 2019 at 3:12 PM Matt Caswell <matt@xxxxxxxxxxx
> <mailto:matt@xxxxxxxxxxx>> wrote:
>
>
>
>     On 10/01/2019 18:09, Ananthu Unnikrishnan wrote:
>     > Hi Matt,
>     >
>     >         Thanks a lot for the reply.
>     >
>     >        After calling ASYNC_pause_job() from the engine, control will
>     transfer to
>     > the place where we start the ASYNC_start_job right? So how can we write
>     the code
>     > to put a trigger on fd in the same thread?
>
>     I'm not saying you can't use threads to do this - only that you need to be
>     careful about using them and must obey the rules.
>
>
>     > If I am wrong  please correct me.
>     > Also if u can suggest where resume operation should be done, it would be
>     helpful.
>
>     Typically the calling application will wait on the fd (e.g. using "select" or
>     similar) until it is readable. Once it sees the fd as readable then it calls
>     ASYNC_start_job() again.
>
>
>     As an aside you may also be interested in this PR that is currently being
>     reviewed:
>
>     https://github.com/openssl/openssl/pull/7573
>
>     This adds an alternative mechanism for signalling other than using fds (i.e. to
>     use a callback instead) and will be available in OpenSSL 3.0 when it gets
>     released.
>
>     Matt
>
>
>
>     >
>     > On Thu, Jan 10, 2019 at 10:11 PM Matt Caswell <matt@xxxxxxxxxxx
>     <mailto:matt@xxxxxxxxxxx>
>     > <mailto:matt@xxxxxxxxxxx <mailto:matt@xxxxxxxxxxx>>> wrote:
>     >
>     >
>     >
>     >     On 10/01/2019 09:39, Ananthu Unnikrishnan wrote:
>     >     > Hi all,
>     >     >
>     >     >         We are not able to access the waitctx address from the job
>     address
>     >     using
>     >     > ASYNC_get_wait_ctx(job) from a thread which starts in the bind
>     section of the
>     >     > dynamic engine. The job address is the same as that we got
>     >     > using ASYNC_get_current_job.  Can anyone help on this?
>     >
>     >     It's very unclear from the your various emails what you are trying to
>     achieve
>     >     and the problem that you are experiencing.
>     >
>     >     From the above it sounds like you are starting threads from inside
>     your dynamic
>     >     engine? ASYNC_JOBs are always local to a thread. When a job is started
>     it is
>     >     associated with the current thread. So ASYNC_get_current_job() returns
>     the job
>     >     associated with the current thread. Similarly if you pause a job,
>     >     ASYNC_pause_job() must be called from that same thread. Finally if you
>     restart a
>     >     paused job it must also be restarted on the same thread.
>     >
>     >     If you are starting new threads from within your engine then extreme
>     care must
>     >     be taken to obey the above rules - otherwise you are likely to get strange
>     >     results.
>     >
>     >     Matt
>     >
>     >
>     >     >
>     >     >          
>     >     >
>     >     > On Tue, Jan 8, 2019 at 11:26 AM Ananthu Unnikrishnan
>     >     <ananthuu741@xxxxxxxxx <mailto:ananthuu741@xxxxxxxxx>
>     <mailto:ananthuu741@xxxxxxxxx <mailto:ananthuu741@xxxxxxxxx>>
>     >     > <mailto:ananthuu741@xxxxxxxxx <mailto:ananthuu741@xxxxxxxxx>
>     <mailto:ananthuu741@xxxxxxxxx <mailto:ananthuu741@xxxxxxxxx>>>> wrote:
>     >     >
>     >     >     Can anyone please help on this? If u need any additional information
>     >     please
>     >     >     let me know.
>     >     >
>     >     >     On Mon, Jan 7, 2019 at 6:25 PM Ananthu Unnikrishnan
>     >     <ananthuu741@xxxxxxxxx <mailto:ananthuu741@xxxxxxxxx>
>     <mailto:ananthuu741@xxxxxxxxx <mailto:ananthuu741@xxxxxxxxx>>
>     >     >     <mailto:ananthuu741@xxxxxxxxx <mailto:ananthuu741@xxxxxxxxx>
>     <mailto:ananthuu741@xxxxxxxxx <mailto:ananthuu741@xxxxxxxxx>>>> wrote:
>     >     >
>     >     >         Hi all,
>     >     >
>     >     >         Adding more details to the previous mail. We have edited the
>     OpenSSL
>     >     >         code for implementing the polling for changed fd's as in
>     OpenSSL speed
>     >     >         command. Attached the code snippet of the same along with
>     this mail.
>     >     >         Mentioned below some observations which found doubtful:
>     >     >
>     >     >         1) We have got prints in ASYNC_FINISH case, before getting
>     print in
>     >     >         ASYNC_PAUSE case. Attaching the log also, so you can understand
>     >     easily. 
>     >     >         2) Shown below the code snippet which we have written to
>     provide a
>     >     write
>     >     >         on fd.
>     >     >
>     >     >         if ((waitctx = ASYNC_get_wait_ctx((ASYNC_JOB *)job)) == NULL) {
>     >     >                 printf("In dynamic engine | waitctx == NULL : %d\n",
>     >     __LINE__);
>     >     >                 return ret;
>     >     >             }
>     >     >
>     >     >             printf("\n----- In dynamic engine | After pausing | job
>     is %lx | 
>     >     >         waitctx in resume job %lx |-----\n", job, waitctx);
>     >     >
>     >     >                 if ((ret = ASYNC_WAIT_CTX_get_fd(waitctx, engine_id,
>     &efd,
>     >     >                                       &custom)) > 0) {
>     >     >                 if (write(efd, &buf, sizeof(uint64_t)) == -1) {
>     >     >                     printf("\nFailed to write\n");
>     >     >                 }
>     >     >             }
>     >     >
>     >     >         Here waitctx is getting NULL when we tried to fetch waitctx
>     using
>     >     >         ASYNC_get_wait_ctx(). We have printed data of sizeof(ASYNC_JOB)
>     >     ie.1176
>     >     >         bytes contained in the job address from the engine,  we got
>     >     correct data
>     >     >         before calling ASYNC_pause() and but got half data zero data
>     when we
>     >     >         tried to access the same address after ASYNC_pause(). 
>     >     >
>     >     >         Attaching both job structure contents before and after calling
>     >     >         ASYNC_pause(). Also, we got the correct data in the same address
>     >     when we
>     >     >         have printed from ssl_start_async_job() before polling starts,
>     >     where we
>     >     >         have started the async_job
>     >     >         . 
>     >     >         The prints starting with "In dynamic engine" are prints
>     inside the
>     >     >         engine, rest prints are from ss/ssl_lib.c. Kindly check this and
>     >     please
>     >     >         point out if anything is wrong somewhere. Thanks in advance.
>     >     >
>     >     >
>     >     >         On Sun, Jan 6, 2019 at 10:30 AM Ananthu Unnikrishnan
>     >     >         <ananthuu741@xxxxxxxxx <mailto:ananthuu741@xxxxxxxxx>
>     <mailto:ananthuu741@xxxxxxxxx <mailto:ananthuu741@xxxxxxxxx>>
>     >     <mailto:ananthuu741@xxxxxxxxx <mailto:ananthuu741@xxxxxxxxx>
>     <mailto:ananthuu741@xxxxxxxxx <mailto:ananthuu741@xxxxxxxxx>>>> wrote:
>     >     >
>     >     >             Hi all,
>     >     >
>     >     >                    We have implemented a dynamic engine and tested
>     in the
>     >     async
>     >     >             mode using OpenSSL speed command. But in a real network
>     >     scenario, we
>     >     >             have seen only starting the async_job(in file ssl/ssl_lib.c,
>     >     >             function: ssl_start_async_job) in the OpenSSL. We
>     haven't seen
>     >     >             polling async_fd's for resuming the job as in
>     apps/speed.c(in
>     >     speed
>     >     >             command case).
>     >     >                     So can anyone please suggest any solution for
>     resuming the
>     >     >             async_jobs in real network scenario? Thanks in advance.
>     >     >
>     >     >
>     >     >
>     >     >
>     >     >
>     >     >         --
>     >     >         With best Regards, 
>     >     >
>     >     >                  Ananthu
>     >     >
>     >     >
>     >     >
>     >     >
>     >     >     --
>     >     >     With best Regards, 
>     >     >
>     >     >              Ananthu
>     >     >
>     >     >
>     >     >
>     >     >
>     >     > --
>     >     > With best Regards, 
>     >     >
>     >     >          Ananthu
>     >     >
>     >     >
>     >     >
>     >     --
>     >     openssl-users mailing list
>     >     To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
>     >
>     >
>     >
>     > --
>     > With best Regards, 
>     >
>     >          Ananthu
>     >
>     >
>     >
>     --
>     openssl-users mailing list
>     To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
>
>
>
> --
> With best Regards, 
>
>          Ananthu
>
>
>
--
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users
-- 
openssl-users mailing list
To unsubscribe: https://mta.openssl.org/mailman/listinfo/openssl-users

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

[Index of Archives]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Security]     [Bugtraq]     [Linux]     [Linux OMAP]     [Linux MIPS]     [ECOS]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux