Skip to main content

Posts

Showing posts with the label fast data load

A long and first trip in our Nexon

 Dear Readers               Hope you all are doing well and celebrated the Diwali!                              3 years back, when we bought the Nexon petrol, it took 3 months to go for a long trip which is around 550-kilometer one way.               Usually, for long trips I prefer to have a halt after 300 kilometers, and I never travelled 500+ kilometer in one go with family. So, we had a halt in between and went to Tiruchendur.               And, we reach Tiruchendur by evening and have a darshan as usual. In the next day we sometimes have a darshan once again and start returning home. However, my spouse friend advised us to have a early morning darshan to avoid the crowd.               So, we went to the temple by 230 AM in the morning, ...

Loading 50 million records in 50 minutes in SQL Server - Super fast data loading

 Greetings to all the readers!!  Yes, it is possible to load 1 million of records in 1 minute time from one SQL server to another.  Create an SSIS package with a Script task.  Add logging for ease of debugging and make all the connections configurable to make it re-usable.  In the script task, we have to use .net script given below. We can define the batch size and   timeout.  By Default, Identity in the destination is reset, in case you have to preserve it, amend the code       accordingly. In my task, I used SQL Server 2017, SSIS 2017 and the source system is SQL 2008 R2  C# using System.Data.SqlClient; class Program {     static void Main()     {         string connectionString = GetConnectionString();         // Open a sourceConnection to the AdventureWorks database.         using (SqlConnection sourceConnection =       ...