RESTlet 2.0 and Vert.x for High volume Integrations

Overview In a typical high volume integration scenario most solutions take a scheduled script approach or SuiteTalk, which works fine. But, in one of the benchmark I discovered that RESTlet could achieve much better performance than that of Scheduled scripts or SuiteTalk. Read below to understand how. For this benchmark I considered a CSV file of about 8k cash sale transactions. The SuiteTalk SuiteCloud Plus Approach The initial integration was performed by SuiteTalk which took about 8-10 hours of time, using Java Client, even though SuiteCloud Plus license parallel end points were used using Java Threads. SuiteScript 2.0 MapReduce Approach I thought of using SuiteScripts, as SuiteScripts seems to be more native to NetSuite infrastructure and could save HTTPS and SOAP overhead. In this approach, I split the CSV files into chunks of 5MB using Java client and then uploaded the multiple CSV files into file cab...

Jan 17, 2016 05:51 pm   -   Author: Prasun Sultania   -   Tags:   NetSuite      Java      Vert.x      SuiteScript 2.0   

Export large number of transactions from NetSuite using SuiteAnalytics Connect & Node.js Streams

Overview:To export million transactions from NetSuite to a FTP based system, after applying some transformation, every month.Assuming the data set size can be around 700MB-1GB for this use case, or porbably even more.Prime Challenges:When you export data from one system to another, there are lot many things to consider, but, lets just focus on challenges that are specific to large number of records.1) TimeExporting large data of 1GB can take considerable amount of time using conventional approaches. 2) RAM availability on client machine.Assuming the machine you have access to, has limited RAM and there are other jobs running on the machine. Consuming even near to a GB of RAM, application can run into process out of memory error. Heap usage can go very high if conventional programming approaches are used. You need a programming model tat makes best use of available RAMApproach1:Using RESTlets/SOAP APIs:Doing a saved search for millions of records can take considerable amount of time us...

Sep 27, 2015 05:39 am   -   Author: Prasun Sultania   -   Tags:   NetSuite      Node.js      Java