Tuesday, May 29, 2012

Export Import SharePoint Site using Web Service

Problem:

While working with SharePoint Online – sometimes it is required to take the backup of the site or to export the site to some other environment, but due to no access to SharePoint server, we end up with writing huge code to migrate the site contents to another site using any client object model.
If server side scenario is considered, we have lots of options to perform a simple site export and import operation , like say – use stsadm commands , use Central Administration , use powershell scripts but for SharePoint Online environment what to do? Well here comes web services in picture , how? We will see in details.

Solution:
SharePoint Web services are already there since MOSS 2007, but mostly not used. Here we are going to use the sites.asmx web service to export and import the sites.
Note that before performing these operations using web services – make sure that you have site collection administrator rights on source as well as target site.
I created a Team site as source site and added some sample data to it , say about 4000 documents + 3000 list items with attachments + some content types + some site columns + some lookup columns + some list and content type workflows.

We can use the ExportWeb function given by the sites.asmx web service which takes some parameters as input – let’s see what those are.
  • 1.       jobName: this is the string parameter and can be set to any string , using this name the cmp files will be created. Please make a note that no special characters are allowed here.
  • 2.       webUrl: url of the site which needs to be exported as cmp files.
  • 3.       Datapath: path where all cmp files will be stored (typically a path of the document library on source site)
  • 4.       Includesecurity: as name suggests – Boolean parameter to include security or not
  • 5.       Overwrite: Boolean parameter to overwrite cmp files on save location
  • 6.       cabSize: size of each cmp file , can be set to 2GB max.
  • Export web function returns an integer after completion , but mostly the description is not available for the return types. 
Here it is :

Value
Description
1
Pending: The operation is in progress.
4
InvalidExportUrl: The site specified in the webUrl is not accessible.
5
ExportFileNoAccess: The location specified in the dataPath is not accessible.
6
ExportWebNoAccess: The user does not have the required permissions to execute this operation successfully. The user MUST have the ManageLists and AddListItems permissions in the solution gallery.
7
ExportError: An error other than the errors listed in this table occurred during exporting the site. An example of a condition that would result in this error is if there was no content on the site to export.
8
UploadDataError: The content migration package file is not uploaded to the server successfully. This can occur if the specified path was invalid, read-only, the caller is not authorized to access it, or the file is already open.

ImportWeb:
While importing the web from cmp files using web services – we can use ImportWeb function of sites.asmx web service.

Important part here is – the target site on which import is to be done – should be created using OOB blank site definition. There can be an error while performing the import operation like – the source site is created using template STS#0 and target site is created using STS#1 , - solution to this is – create target site collection without any template – select the select template later option while creating target the site collection.
ImportWeb function take some parameters let’s see what those are
  • 1.       jobName: string parameter , can be set to anything – but no special characters should be included.
  • 2.       webUrl: url of target web.
  • 3.       dataFiles: this is array of string containing url of all cmp files. Example : If there are 10 cmp files in the source location – then dataFiles parameter will be any array having urls of 10 files in ascending order.
  • 4.       logPath: path where log file can be created. This can be passed as null. (typically a url of any document library)
  • 5.       includesecurity: to include security or not
  • 6.       overwrite: overwrite log files.
Similar to ExportWeb function – ImportWeb function also return an interger on completion.
Here are the details

Value
Description
1
Pending: The operation is in progress.
2
GenericError: An error other than the errors listed in this table occurred importing the site.
4
InvalidImportUrl: The site specified in the webUrl is not accessible.
5
ImportFileNoAccess: At least one location specified in dataFiles is not accessible.
6
ImportWebNoAccess: The user has insufficient permission to import to the location specified in webUrl.
8
mportWebNotEmpty: The location specified by webUrl corresponds to an existing site that is not a blank site.
11
LogFileNoAccess: The location specified by logPath is not accessible.

Ok , so after lots of theory – here is the simple console application example which I tried on my environment successfully.

static void Main(string[] args)
{
    string sourcesiteurl = "http://sourcesitecollection/";
    string targetsiteurl = "http://targetblanksitecollection/";
    string svcUrl = "_vti_bin/sites.asmx";

    Sites sites = new Sites();
    NetworkCredential credentials = new NetworkCredential("username", "password", "domain");
    sites.Credentials = credentials;
    sites.Timeout = 600000;

    Console.WriteLine("Choose..1. Export , 2. Import");
    string responce = Console.ReadLine();

    if (responce.Equals("1"))
    {
       sites.Url = string.Format("{0}{1}", sourcesiteurl, svcUrl);
       try
       {
         int result = sites.ExportWeb("MyExportTest", sourcesiteurl, "http://sourcesite/Exports", true, true, true, 1024);
         Console.WriteLine("done with result - " + result.ToString());
       }
       catch (Exception ex)
       {
          Console.WriteLine(ex.Message);
          Console.WriteLine();
          Console.WriteLine(ex.StackTrace);
       }
     }

     else if (responce.Equals("2"))
     {
         sites.Url = string.Format("{0}{1}", targetsiteurl, svcUrl);
         string[] dataFiles = new string[] { "http://sourcesite/Exports/MyExportTest.cmp" };
         
          try
          {
            int result = sites.ImportWeb("MyExportTest", targetsiteurl, dataFiles, null, true, true);
            Console.WriteLine("done with result - " + result.ToString());
          }
          catch (Exception ex)
          {
            Console.WriteLine(ex.Message);
            Console.WriteLine();
            Console.WriteLine(ex.StackTrace);
          }
       }
     Console.ReadLine();
   }



12 comments:

  1. Very good post, Bhushan. I had to do quite a bit of digging to find out how you resolved the blank site problem that you pointed out. Have you noticed when you do an import, that all timestamps are current and not what was on the original site? Do you happen to know how to get around this? Also, can you point me in the right direction to get all the other components imported over, like workflows?

    ReplyDelete
  2. About preserving timestamps of items , AFAIK using this method you wont be able to preserve those as you are adding new content on exiting blank site so they are considered as new items. and about workflows - list and content type workflows should gets migrated but If they refer to List GUIDs then you should be changing those on target site , also you wont be able to export these - alerts, audit trail, recycle bin items, security state, workflow tasks/state are not exported.
    different approaches compared here , you can refer - http://www.sharepointnutsandbolts.com/2007/10/stsadm-export-content-deployment.html

    ReplyDelete
  3. Import Return code 8?

    Hello Bhushan,
    Great post this helped me a lot. SPO Export of a site collection to cmp work flawless with your help. But I struggle with the Import of the exported site colletion cmp to a new site under the same site collection. Created a new site with the same template (TeamSite). No I get always the return code 8 (site is not empty?) Even if I delete everything still the same error? Would be very greatful for any advice.
    Thank you
    Max

    ReplyDelete
  4. Hi Max,
    Glad that this post helped you :)
    About the error which you mentioned - As you said that you have created Team site , but import always is done on the Empty site so please make sure that you create the target site collection / new site using blank site definition.

    ReplyDelete
  5. Hi Bhushan,
    thank you for your response. I tried to do this and created a new targetsite with the blank site template but than I get the error:
    "Cannot import site. The exported site is based on the template STS#0 but the destination site is based on the template STS#1. You can import sites only into sites that are based on same template as the exported site."
    Is there another way to create a working blank and empty target site for Import?
    Also backup worked perfect for smaller sites (60-200 mb) so far but I tried this now for a bigger site with about 4-5 GB data. It looks ok and returns 1 but then it takes forever to complete (12+ hours and still not finished)?
    Thank you very much in advance for your help.
    Max

    ReplyDelete
  6. Hi Max,
    Should have worked with blank site definition , how about creating the target site collection without any site template by following steps below and then perform the import ?
    Open Central Administration > Application Management >Create Site Collection > Select Custom Tab > Select - Select Template Later

    about your second question - Honestly I didn't try to use this approach with larger sites but as all the .cmp files resides in the one of the library of site so content database size would be getting much larger and so similarly reading those .cmp files might be time consuming.

    ReplyDelete
  7. Hello Bhushan,
    thank you for your help. I will try it with select template later.
    I solved my problem with a large sharepoint site export (3-4 GB)! Anything above 100 MB for cab-size is not working and ends with an error not enough storage for compressing a large file!! I ended with a cab-size of 96 MB. So you get a lot of cmp-files but it appears to be the only way.
    Thanks,
    Max

    ReplyDelete
  8. In am receiving 3 as the result in ImportWeb method. Please specif the error

    ReplyDelete
  9. @Bhushan Import works now with your hint to create site with template later option :).
    @Paramdeep here is a more complete list of error codes.

    List of response codes:
    0=Success
    1=Pending
    2=GenericError
    3=InvalidArgument
    4=InvalidImportUrl
    5=ImportFileNoAccess
    6=ImportWebNoAccess
    7=ImportWebCantCreate
    8=ImportWebNotEmpty
    9=PrepareDataFolderError
    10=DownloadDataError
    11=LogFileNoAccess
    12=ImportError
    13=CleanupError

    warm regards,
    Max

    ReplyDelete
  10. Hello Bushan,

    just run into another Import issue. How to get the right array of string containing url of all the numbered cmp files from the document library the best way with c#? Due to the backup of a large site I have up to 40 cmp files.

    Thank you,
    Max

    ReplyDelete
  11. Hi Bushan,
    i run your script against SharePoint online (SharePoint 2013) with some authentication modifications.
    At the ExportWeb Method i always get warning number 5. I can't save the Export file to a document library.
    How did you solved this Problem?

    Greetings
    Christian

    ReplyDelete
  12. Hi Christian,
    Can you make sure that you have site collection rights or at least write permissions on the document library where you are trying to save your exported files?

    ReplyDelete