We often get questions from customers about how to share a drive with read-write access among multiple role instances. A common scenario is that of a content repository for multiple web servers to access and store content. An Azure drive is similar to a traditional disk drive in that it may only be mounted read-write on one system. However using SMB, it is possible to mount a drive on one role instance and then share that out to other role instances which can map the network share to a drive letter or mount point.
In this blog post we?ll cover the specifics on how to set this up and leave you with a simple prototype that demonstrates the concept. We?ll use an example of a worker role (referred to as the server) which mounts the drive and shares it out and two other worker roles (clients) that map the network share to a drive letter and write log records to the shared drive.
Service Definition on the Server role
The server role has TCP port 445 enabled as an internal endpoint so that it can receive SMB requests from other roles in the service. This done by defining the endpoint in the ServiceDefinition.csdef as follows
<Endpoints> <InternalEndpoint name="SMB" protocol="tcp" port="445" /> </Endpoints>
Now when the role starts up, it must mount the drive and then share it. Sharing the drive requires the Server role to be running with administrator privileges. Beginning with SDK 1.3 it?s possible to do that using the following setting in the ServiceDefinition.csdef file.
<Runtime executionContext="elevated"> </Runtime> Mounting the drive and sharing it
When the server role instance starts up, it first mounts the Azure drive and executes shell commands to
- Create a user account for the clients to authenticate as. The user name and password are derived from the service configuration.
- Enable inbound SMB protocol traffic through the role instance firewall
- Share the mounted drive with the share name specified in the service configuration and grant the user account previously created full access. The value for path in the example below is the drive letter assigned to the drive.
Here?s snippet of C# code that does that.
String error; ExecuteCommand("net.exe", "user " + userName + " " + password + " /add", out error, 10000); ExecuteCommand("netsh.exe", "firewall set service type=fileandprint mode=enable scope=all", out error, 10000); ExecuteCommand("net.exe", " share " + shareName + "=" + path + " /Grant:" + userName + ",full", out error, 10000);
The shell commands are executed by the routine ExecuteCommand.
public static int ExecuteCommand(string exe, string arguments, out string error, int timeout) { Process p = new Process(); int exitCode; p.StartInfo.FileName = exe; p.StartInfo.Arguments = arguments; p.StartInfo.CreateNoWindow = true; p.StartInfo.UseShellExecute = false; p.StartInfo.RedirectStandardError = true; p.Start(); error = p.StandardError.ReadToEnd(); p.WaitForExit(timeout); exitCode = p.ExitCode; p.Close(); return exitCode; }
We haven?t touched on how to mount the drive because that is covered in several places including here.
Mapping the network drive on the client
When the clients start up, they locate the instance of the SMB Server and then identify the address of the SMB endpoint on the server. Next they execute a shell command to map the share served by the SMB server to a drive letter specified by the configuration setting localpath. Note that sharename, username and password must match the settings on the SMB server.
var server = RoleEnvironment.Roles["SMBServer"].Instances[0]; machineIP = server.InstanceEndpoints["SMB"].IPEndpoint.Address.ToString();machineIP = "\\\\" + machineIP + "\\"; string error; ExecuteCommand("net.exe", " use " + localPath + " " + machineIP + shareName + " " + password + " /user:"+ userName, out error, 20000);
Once the share has been mapped to a local drive letter, the clients can write whatever they want to the share, just as they would to a local drive.
Note: Since the clients may come up before the server is ready, the clients may have to retry or alternatively poll the server on some other port for status before attempting to map the drive. The prototype retries in a loop until it succeeds or times out.
Enabling High Availability
With a single server role instance, the file share will be unavailable when the role is being upgraded. If you need to mitigate that, you can create a few warm stand-by instances of the server role thus ensuring that there is always one server role instance available to share the Azure Drive to clients.
Another approach would be to make each of your roles a potential host for the SMB share. Each role instance could potentially run an SMB service, but only one of them would get the mounted Azure Drive behind SMB service. The roles can then iterate over all the role instances attempting to map the SMB share with each role instance. The mapping will succeed when the client connects to the instance that has the drive mounted.
Another scheme is to have the role instance that successfully mounted the drive inform the other role instances so that the clients can query to find the active server instance.
Note: The high availability scenario is not captured in the prototype but is feasible using standard Azure APIs.
Sharing Local Drives within a role instance
It?s also possible to share a local resource drive mounted in a role instance among multiple role instances using similar steps. The key difference though is that writes to the local storage resource are not durable while writes to Azure Drives are persisted and available even after the role instances are shutdown.
Dinesh Haridas
Sample Code
Here?s the code for the Server and Client in its entirety for easy reference.
Server ? WorkerRole.cs
This file contains the code for the SMB server worker role. In the OnStart() method, the role instance initializes tracing before mounting the Azure Drive. It gets the settings for storage credentials, drive name and drive size from the Service Configuration. Once the drive is mounted, the role instance creates a user account, enables SMB traffic through the firewall and then shares the drive. These operations are performed by executing shell commands using the ExecuteCommand() method described earlier. For simplicity, parameters like account name, password and the share name for the drive are derived from the Service Configuration.
using System; using System.Collections.Generic; using System.Diagnostics; using System.Linq; using System.Net; using System.Threading; using Microsoft.WindowsAzure; using Microsoft.WindowsAzure.Diagnostics; using Microsoft.WindowsAzure.ServiceRuntime; using Microsoft.WindowsAzure.StorageClient; namespace SMBServer { public class WorkerRole : RoleEntryPoint { public static string driveLetter = null; public static CloudDrive drive = null; public override void Run() { Trace.WriteLine("SMBServer entry point called", "Information"); while (true) { Thread.Sleep(10000); } } public override bool OnStart() { // Set the maximum number of concurrent connections ServicePointManager.DefaultConnectionLimit = 12; // Initialize logging and tracing DiagnosticMonitorConfiguration dmc = DiagnosticMonitor.GetDefaultInitialConfiguration(); dmc.Logs.ScheduledTransferLogLevelFilter = LogLevel.Verbose; dmc.Logs.ScheduledTransferPeriod = TimeSpan.FromMinutes(1); DiagnosticMonitor.Start("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString", dmc); Trace.WriteLine("Diagnostics Setup complete", "Information"); CloudStorageAccount account = CloudStorageAccount.Parse(RoleEnvironment.GetConfigurationSettingValue("StorageConnectionString")); try { CloudBlobClient blobClient = account.CreateCloudBlobClient(); CloudBlobContainer driveContainer = blobClient.GetContainerReference("drivecontainer"); driveContainer.CreateIfNotExist(); String driveName = RoleEnvironment.GetConfigurationSettingValue("driveName"); LocalResource localCache = RoleEnvironment.GetLocalResource("AzureDriveCache"); CloudDrive.InitializeCache(localCache.RootPath, localCache.MaximumSizeInMegabytes); drive = new CloudDrive(driveContainer.GetBlobReference(driveName).Uri, account.Credentials); try { drive.Create(int.Parse(RoleEnvironment.GetConfigurationSettingValue("driveSize"))); } catch (CloudDriveException ex) { Trace.WriteLine(ex.ToString(), "Warning"); } driveLetter = drive.Mount(localCache.MaximumSizeInMegabytes, DriveMountOptions.None); string userName = RoleEnvironment.GetConfigurationSettingValue("fileshareUserName"); string password = RoleEnvironment.GetConfigurationSettingValue("fileshareUserPassword"); // Modify path to share a specific directory on the drive string path = driveLetter; string shareName = RoleEnvironment.GetConfigurationSettingValue("shareName"); int exitCode; string error; //Create the user account exitCode = ExecuteCommand("net.exe", "user " + userName + " " + password + " /add", out error, 10000); if (exitCode != 0) { //Log error and continue since the user account may already exist Trace.WriteLine("Error creating user account, error msg:" + error, "Warning"); } //Enable SMB traffic through the firewall exitCode = ExecuteCommand("netsh.exe", "firewall set service type=fileandprint mode=enable scope=all", out error, 10000); if (exitCode != 0) { Trace.WriteLine("Error setting up firewall, error msg:" + error, "Error"); goto Exit; } //Share the drive exitCode = ExecuteCommand("net.exe", " share " + shareName + "=" + path + " /Grant:" + userName + ",full", out error, 10000); if (exitCode != 0) { //Log error and continue since the drive may already be shared Trace.WriteLine("Error creating fileshare, error msg:" + error, "Warning"); } Trace.WriteLine("Exiting SMB Server OnStart", "Information"); } catch (Exception ex) { Trace.WriteLine(ex.ToString(), "Error"); Trace.WriteLine("Exiting", "Information"); throw; } Exit: return base.OnStart(); } public static int ExecuteCommand(string exe, string arguments, out string error, int timeout) { Process p = new Process(); int exitCode; p.StartInfo.FileName = exe; p.StartInfo.Arguments = arguments; p.StartInfo.CreateNoWindow = true; p.StartInfo.UseShellExecute = false; p.StartInfo.RedirectStandardError = true; p.Start(); error = p.StandardError.ReadToEnd(); p.WaitForExit(timeout); exitCode = p.ExitCode; p.Close(); return exitCode; } public override void OnStop() { if (drive != null) { drive.Unmount(); } base.OnStop(); } } }
Client ? WorkerRole.cs
This file contains the code for the SMB client worker role. The OnStart method initializes tracing for the role instance. In the Run() method, each client maps the drive shared by the server role using the MapNetworkDrive() method before writing log records at ten second intervals to the share in a loop.
In the MapNetworkDrive() method the client first determines the IP address and port number for the SMB endpoint on the server role instance before executing the shell command net use to connect to it. As in the case of the server role, the routine ExecuteCommand() is used to execute shell commands. Since the server may start up after the client, the client retries in a loop sleeping 10 seconds between retries and gives up after about 17 minutes. Between retries the client also deletes any stale mounts of the same share.
using System; using System.Collections.Generic; using System.Diagnostics; using System.Linq; using System.Net; using System.Threading; using Microsoft.WindowsAzure; using Microsoft.WindowsAzure.Diagnostics; using Microsoft.WindowsAzure.ServiceRuntime; using Microsoft.WindowsAzure.StorageClient; namespace SMBClient { public class WorkerRole : RoleEntryPoint { public const int tenSecondsAsMS = 10000; public override void Run() { // The code here mounts the drive shared out by the server worker role // Each client role instance writes to a log file named after the role instance in the logfile directory Trace.WriteLine("SMBClient entry point called", "Information"); string localPath = RoleEnvironment.GetConfigurationSettingValue("localPath"); string shareName = RoleEnvironment.GetConfigurationSettingValue("shareName"); string userName = RoleEnvironment.GetConfigurationSettingValue("fileshareUserName"); string password = RoleEnvironment.GetConfigurationSettingValue("fileshareUserPassword"); string logDir = localPath + "\\" + "logs"; string fileName = RoleEnvironment.CurrentRoleInstance.Id + ".txt"; string logFilePath = System.IO.Path.Combine(logDir, fileName); try { if (MapNetworkDrive(localPath, shareName, userName, password) == true) { System.IO.Directory.CreateDirectory(logDir); // do work on the mounted drive here while (true) { // write to the log file System.IO.File.AppendAllText(logFilePath, DateTime.Now.TimeOfDay.ToString() + Environment.NewLine); Thread.Sleep(tenSecondsAsMS); } } Trace.WriteLine("Failed to mount" + shareName, "Error"); } catch (Exception ex) { Trace.WriteLine(ex.ToString(), "Error"); throw; } } public static bool MapNetworkDrive(string localPath, string shareName, string userName, string password) { int exitCode = 1; string machineIP = null; while (exitCode != 0) { int i = 0; string error; var server = RoleEnvironment.Roles["SMBServer"].Instances[0]; machineIP = server.InstanceEndpoints["SMB"].IPEndpoint.Address.ToString(); machineIP = "\\\\" + machineIP + "\\"; exitCode = ExecuteCommand("net.exe", " use " + localPath + " " + machineIP + shareName + " " + password + " /user:" + userName, out error, 20000); if (exitCode != 0) { Trace.WriteLine("Error mapping network drive, retrying in 10 seoconds error msg:" + error, "Information"); // clean up stale mounts and retry ExecuteCommand("net.exe", " use " + localPath + " /delete", out error, 20000); Thread.Sleep(10000); i++; if (i > 100) break; } } if (exitCode == 0) { Trace.WriteLine("Success: mapped network drive" + machineIP + shareName, "Information"); return true; } else return false; } public static int ExecuteCommand(string exe, string arguments, out string error, int timeout) { Process p = new Process(); int exitCode; p.StartInfo.FileName = exe; p.StartInfo.Arguments = arguments; p.StartInfo.CreateNoWindow = true; p.StartInfo.UseShellExecute = false; p.StartInfo.RedirectStandardError = true; p.Start(); error = p.StandardError.ReadToEnd(); p.WaitForExit(timeout); exitCode = p.ExitCode; p.Close(); return exitCode; } public override bool OnStart() { // Set the maximum number of concurrent connections ServicePointManager.DefaultConnectionLimit = 12; DiagnosticMonitorConfiguration dmc = DiagnosticMonitor.GetDefaultInitialConfiguration(); dmc.Logs.ScheduledTransferLogLevelFilter = LogLevel.Verbose; dmc.Logs.ScheduledTransferPeriod = TimeSpan.FromMinutes(1); DiagnosticMonitor.Start("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString", dmc); Trace.WriteLine("Diagnostics Setup comlete", "Information"); return base.OnStart(); } } }
ServiceDefinition.csdef
<?xml version="1.0" encoding="utf-8"?> <ServiceDefinition name="AzureDemo" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceDefinition"> <WorkerRole name="SMBServer"> <Runtime executionContext="elevated"> </Runtime> <Imports> <Import moduleName="Diagnostics" /> </Imports> <ConfigurationSettings> <Setting name="StorageConnectionString" /> <Setting name="driveName" /> <Setting name="driveSize" /> <Setting name="fileshareUserName" /> <Setting name="fileshareUserPassword" /> <Setting name="shareName" /> </ConfigurationSettings> <LocalResources> <LocalStorage name="AzureDriveCache" cleanOnRoleRecycle="true" sizeInMB="300" /> </LocalResources> <Endpoints> <InternalEndpoint name="SMB" protocol="tcp" port="445" /> </Endpoints> </WorkerRole> <WorkerRole name="SMBClient"> <Imports> <Import moduleName="Diagnostics" /> </Imports> <ConfigurationSettings> <Setting name="fileshareUserName" /> <Setting name="fileshareUserPassword" /> <Setting name="shareName" /> <Setting name="localPath" /> </ConfigurationSettings> </WorkerRole> </ServiceDefinition>
ServiceConfiguration.cscfg
<?xml version="1.0" encoding="utf-8"?> <ServiceConfiguration serviceName="AzureDemo" xmlns="http://schemas.microsoft.com/ServiceHosting/2008/10/ServiceConfiguration" osFamily="1" osVersion="*"> <Role name="SMBServer"> <Instances count="1" /> <ConfigurationSettings> <Setting name="StorageConnectionString" value="DefaultEndpointsProtocol=http;AccountName=yourstorageaccount;AccountKey=yourkey" /> <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="DefaultEndpointsProtocol=https;AccountName=yourstorageaccount;AccountKey=yourkey" /> <Setting name="driveName" value="drive2" /> <Setting name="driveSize" value="1000" /> <Setting name="fileshareUserName" value="fileshareuser" /> <Setting name="fileshareUserPassword" value="SecurePassw0rd" /> <Setting name="shareName" value="sharerw" /> </ConfigurationSettings> </Role> <Role name="SMBClient"> <Instances count="2" /> <ConfigurationSettings> <Setting name="Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString" value="DefaultEndpointsProtocol=https;AccountName=yourstorageaccount;AccountKey=yourkey" /> <Setting name="fileshareUserName" value="fileshareuser" /> <Setting name="fileshareUserPassword" value="SecurePassw0rd" /> <Setting name="shareName" value="sharerw" /> <Setting name="localPath" value="K:" /> </ConfigurationSettings> </Role> </ServiceConfiguration>
Cat Power January Jones Christina DaRe Malin Akerman Melissa Joan Hart
No comments:
Post a Comment