Considerations for optimum performance

Review the considerations to capture the performance metrics of IQ Bot standard forms with a new Model containing more number of forms or table fields.

Test summary

  • Performance test was run for 10000 COVID Form documents (two pages per document = 20000 pages) on a single node IQ Bot server using a single Bot Agent (using IQ Bot Extraction package). This test was conducted to benchmark the performance metrics such as processing time, validation, and documents processed (#successful processed pages/documents and #failed pages/documents).
  • New Model (WalgreenComplete 3260b299-68b5-4118-87df-b313bd5c30f6) with 26 Form Fields & 8 Tables fields are trained and used for extraction.
  • With new Model (WalgreenComplete 3260b299-68b5-4118-87df-b313bd5c30f6), all the documents were moving to validation during extraction.
  • Total time take to process 20000 pages using single Bot Agent was 25 hours, 21 minutes, & 42 seconds. This was based on IQ Bot On-Premises Build 12350.
  • Average time take to process single page using IQ Bot Extraction package on single Bot Agent is approximate 4.56 seconds.
  • During our test execution, the requests were served by Microsoft standard forms services hosted in us-west2 region.
  • As confirmed, Microsoft team TPS (Transactions Per Second) supported by us-west2 region was 100 TPS
Note: KPI (Key Performance Indicators) like processing time, accuracy might vary based on the regions where Microsoft standard forms service is hosted and its transactions per second (TPS) rate.

Infrastructure configurations

The following table lists the infrastructure configurations:
Server Parameters Values
Server 1 [Web Server] Control Room Windows Server 2019 → 8vCPU, 16 GB RAM, 512 GB Disk [Rackspace Platform]-uee
Server 2 [Database] Database Windows Server 2019 → 8vCPU, 16 GB RAM, 512 GB Disk [Rackspace Platform]
Server 3 [Application Server] IQ Bot Windows Server 2019 → 8vCPU, 16 GB RAM, 512 GB Disk [Rackspace Platform]
Server 4 [Client] Bot Agent Windows Server 2019 → 4vCPU, 8 GB RAM, 256 GB Disk [Rackspace Platform]

Product configurations

The following table lists the product configurations:
Parameters Values
Automation Anywhere Enterprise Automation 360 Build 12313
Bot Agent 21.80.10273
IQ Bot A360.24.28441-20220311
IQ Bot Extraction package 1.0.0-20220307-024320
Database Microsoft SQL Server Developer (64-bit)
Protocol HTTPS
Network services Non-Active Directory

Data configurations

The following table lists the data configurations:
Parameters Values
Domain Standard forms
Provider Forms Extractor Type 1
Model WalgreenComplete 3260b299-68b5-4118-87df-b313bd5c30f6
Document Type Multi page PDF document
Pages Two pages per document
Form Fields 26

[A2Date, A2PatientSign, Address, Age, B2Date, B2Sign, City, DOB, Email, FirstName, LTCFName, LastName, Phone, PrintName, Race-American, Race-Asian, Race-Black, Race-Native, Race-Other, Race-UnableToReport, Race-Unknown, Race-White, RefusedInformation, State, VaccinationType, Zipcode]
Table Fields 8 tables, 23 table fields

Table1: DriversLicense[Issuance, LicenseNo]Table2: Ethnicity[Hispanic, NotHispanic, UnableToReport, Unknown]Table3: Gender[Female, Male]Table4: InsuranceCard[Card Details, Medical Card, Pharmacy Card]Table5: MedicareDetails[Medicare, Medicare PartB]Table6: PatientCardHolder[DontKnow, No, Question, Yes]Table7: PatientType[Resident, StaffMember]Table8: ScreeningQuestions[DontKnow, No, Question, Yes]
Configuration COVID forms
# of Documents 10000
# of Pages 20000
Language English

Test results

The following table lists the test results:
Parameters Values
Total Documents Uploaded 10000
Total Pages Uploaded 20000
Total Processed documents 10000
Total Processed pages 20000
Success 10000 documents
Pending Review (Validation) 0
Accuracy 100%
Processing Time 25 hours, 21 minutes, and 42 seconds

Resource consumption data

The following table provides the resource consumption data:
Parameters CPU Utilization in % Memory Utilization in %
IQ Bot Average: 3.4

Maximum: 26.8

 Average: 70.9

Maximum: 78.9
Control Room Average: 1.4

Maximum: 14.5

 Average: 83.3

Maximum: 84.8
Database Average: 3.3

Maximum: 20.6

 Average: 91.5

Maximum: 94.1
Bot Agent (Extraction) Average: 4.8

Maximum: 48.9

 Average: 52.4

Maximum: 58.7