How to run bulk commands in parallel

Hello, I am trying to learn how to use the bulk framework in nuxeo based on this document https://doc.nuxeo.com/nxdoc/1010/bulk-action-framework/.

But until now I could not find how to process documents in parallel.

For example, I defined the following action (just to test) which allows to sleep one second for each document,

...
public class MyComputation extends AbstractBulkComputation {
...
@Override
    protected void compute(CoreSession session, List<String> ids, Map<String, Serializable> properties) {
    for (DocumentModel doc : loadDocuments(session, ids)) {
            try {
                Thread.sleep(1000);
            } catch (InterruptedException e) {
                e.printStackTrace();
            }
    }
}
...
}

By performing this action on 10 documents, I will have a Bulkstatus with the COMPLETED state only after 10 seconds.

How can I parallelize the processing so that I will have an answer in one second in my example ?

Thanks.

0 votes

1 answers

809 views

ANSWER



I think we can define parallelism in streamproceesor extenision point using defaultConcurrency and defaultPartitions parameters.

For example the following configuration allows to launch 6 buckets in parallel

 <policy name="syllabsIndexing" maxRetries="10" delay="1s" maxDelay="60s" continueOnFailure="true" />  
</streamProcessor>

0 votes