Azure vs AWS comparison: Which Works Best for Serverless Architecture?

An increasing number of modern companies are striving to optimise the cost of their IT infrastructure, including those who’ve already adopted cloud-based services. So cloud vendors are adapting to the challenge by changing their services to offer flexibility and more options for human involvement. The result of this is serverless architecture, and I’ve taken it upon myself to find out which cloud service is the best option for building serverless architecture. Here’s my Azure vs AWS comparison.

Why should I go serverless?

AWS vs Azure

There are a few key benefits to going serverless:

  • no server management required
  • flexibility to scale
  • you pay as you use
  • automation is highly available

I’m going to analyse two cloud platforms used in the serverless architecture here: Azure vs AWS. I’ll offer a comparison of the services provided by each vendor, the price for the end user and their performance – especially in terms of execution speed etc.

​Azure vs AWS comparison: cost, performance and speed​

One scenario we’ll consider here is building a communication engine for messaging. As you’ll know, a serverless approach doesn’t guarantee end-to-end message delivery, and we’d need to take this into account when building our application. The cheapest way to facilitate continuous messaging on the Azure platform is by using Azure Service Bus, with a solution architecture that looks a little something like this:
AWS vs Azure

If we’re thinking about using a similar application in AWS, we can use Kinesis Stream instead of Service Bus, with the following solution architecture:
AWS vs Azure

​Our next step is to generate simple code and input it into Azure Functions or Lambda. Here are a few examples:

AWS Lambda code

console.log('Loading function'); 

exports.handler = function myFunc(event, context) { 
    //console.log(JSON.stringify(event, null, 2)); 

    event.Records.forEach(function(record) { 
        // Kinesis data is base64 encoded so decode here 

        var payload = new Buffer(record.kinesis.data, 'base64').toString('ascii'); 

        console.log('Decoded payload:', payload); 
    }); 
}; 

function myFunc() { 
  console.log('Data was processed');; 
} 

setTimeout(myFunc, 3000);​ ​

Code for Azure Functions

​​using System; 

using System.Threading.Tasks; 

public static void Run(string myQueueItem, ILogger log) { 
    var result = myQueueItem + " new message via azure function"; 

    System.Threading.Thread.Sleep(3000); 

    log.LogInformation($"C# ServiceBus queue trigger function processed message: {result}"); 
} 

We’d also need to develop code for generating messages. Both examples are based on Net core 2.2 C# language. Here are some examples:

For Azure Service Bus queue

using System; 

using System.Collections.Generic; 

using System.Text; 

using System.Threading.Tasks; 

using Microsoft.Azure.ServiceBus; 

using Microsoft.Azure.ServiceBus.Management; 

namespace azure { 
    class Program { 
        private static readonly string _queueName = "myqueue"; 

        private static IQueueClient _queue; 

        private static readonly int _messagesToSend = 10000; 

        static void Main(string[] args) { 
            MainAsync().GetAwaiter().GetResult(); 
        } 

        static async Task MainAsync() { 
            Console.ForegroundColor = ConsoleColor.Cyan; 

            Console.WriteLine($"Producer ({Configs.PID})"); 

            await Bootstrap.Lab01Async(_queueName); 

            _queue = new QueueClient(Configs.SbFailoverConnectionString, _queueName); 

            var tasks = new List(); 

            for (var i = 0; i < _messagesToSend; i++) { 
                var messageBody = $"pid {Configs.PID} message {i}"; 

                var message = new Message(Encoding.UTF8.GetBytes(messageBody)); 

                Console.WriteLine($"Sending message: {messageBody}"); 

                tasks.Add(_queue.SendAsync(message)); 
            } 
            Task.WaitAll(tasks.ToArray()); 

            await _queue.CloseAsync(); 

            Console.WriteLine("Done!"); 

            Console.ResetColor(); 
        } 
    } 
} 

​For AWS Kinesis Stream​

using System; 

using System.Collections.Generic; 

using System.Diagnostics; 

using System.IO; 

using System.Text; 

using System.Threading.Tasks; 

using Amazon.Kinesis; 

using Amazon.Kinesis.Model; 

using Amazon.Runtime; 

using Amazon.SQS; 

using Amazon.SQS.Model; 

namespace AWS { 
    class Program { 
        private static readonly int _messagesToSend = 10000; 

        private static readonly string QueueUrl = "https://..."; 

        static async System.Threading.Tasks.Task Main(string[] args) { 
            Console.WriteLine("Starting pushing ..."); 

            var awsCreds = new BasicAWSCredentials("….", "…"); 

            AmazonKinesisClient kinesisClient = new AmazonKinesisClient(awsCreds, Amazon.RegionEndpoint.USEast2); 

            var tasks = new List(); 

            for (var i = 0; i < _messagesToSend; i++) { 
                var messageBody = $"pid {Process.GetCurrentProcess().Id} message {i}"; 

                var message = Encoding.UTF8.GetBytes(messageBody); 

                using (MemoryStream memoryStream = new MemoryStream(message)) { 
                    PutRecordRequest requestRecord = new PutRecordRequest(); 

                    requestRecord.StreamName = "servicebus"; 

                    requestRecord.PartitionKey = "url-response-times"; 

                    requestRecord.Data = memoryStream; 

                    Console.WriteLine($"Sending message: {messageBody}"); 

                    tasks.Add(kinesisClient.PutRecordAsync(requestRecord)); 
                } 
            } 
            Task.WaitAll(tasks.ToArray()); 

            Console.WriteLine("Messages successfully sent"); 
        } 
    } 
} 

Now it’s time to run an application and measure the outcomes, to determine which cloud platform is the best for building a serverless solution. The test results are summarised in the table below.

​​Type​ ​​Azure​ ​​AWS​
​​Components for testing​ ​​Azure function, Application insights, ServiceBus​ ​​Kinesis, Lambda, Cloud Watch
​​Number of messages​ ​​10000​ ​​10000​
​​Execution duration​ ​​4 min​ ​​6 min​
​​Errors during the execution​ ​​0%​ ​​4%​
​​Average execution time​ ​​3.2 sec​ ​​1.95 sec​
​​The Price of solution​ ​​0.4 $ per 10k messages executed ​​2.10 $ per 10k messages executed​
​​Cold start timeout​ ​​At least one pre-warmed instance (Premium plan)​ ​​Any​
​​Message delivery guarantee​ ​​100%​ ​​100%​
​​Size​ ​​Automatically depends on the load ​​On-demand​
​​Tools (languages)​ ​​JavaScript, C#, Python, PHP​ ​​NodeJS, Python, Java, C#​
​​Monitor (Logs)​ ​​Yes​ ​​Yes​
​​Real-time logs​ ​​Yes​ ​​No​
​​Supporting VNET​ ​​Yes​ ​​No​
​​Max number of functions​ ​​No limit​ ​​No limit​
​​Concurrent execution​ ​​No limit​ ​​1000 parallel execution​
​​Orchestration​ ​​Logic apps + Durable functions​ ​​AWS step functions​
​​Idle time​ ​​No limit​ ​​27 minutes (active time)​
​​Debugging​ ​​Support Visual Studio​ ​​SAM CLI tool which allows you to run/test Lambda but does not support C# listener​
​​Hosting​ ​​Cloud or On-premises​ ​​Cloud​
Timeout Up to 10 min 5 min

Conclusion

Both AWS Lambda and Azure Functions provide a mechanism to execute code (functions) without setting up a server.  They both support creating/editing functions, purely from their portals, or you can develop functions in Visual Studio and deploy to the cloud.  Both offer numerous triggers to kick off the function, for example, document creation, queue messages and cloud events etc. Both support the use of environment variables for configuration management. Basic monitoring of functions is also included in both.

So, as you can see, Azure Functions and AWS Lambda both have pretty good results. But our experiment did demonstrate additional benefits to using Azure Functions, such as:

  1. ​It didn’t produce any errors during execution and message processing.
  2. It has a rich set of tools for real-time monitoring and debugging when using native tools like Visual Studio.
  3. Azure Functions has more opportunities to bind different sources and create a pipeline.
  4. The execution period in Azure Functions can be up to 10 min.
  5. Azure Functions has less cold-starting timeout because the premium plan offers a pre-warmed setup.

What I’ve tried to do here is describe my experience of using similar services from different vendors, to give the best possible Azure vs AWS comparison. From my point of view, either option is perfectly valid, but I’d strongly recommend analysing the services and their potential gaps, to find the best fit for your specific needs.

From my perspective, I’d use the Azure platform if you’re looking for rich debugging tools or you prefer to use Visual Studio for developing. In these instances, Azure is probably the best bet for your serverless application.

On the other hand, if you’re looking for a serverless application that can be scaled up or down on demand, AWS might offer better control and scale-manageability.

I’d love to hear your experiences of using both tools. Which one produced the best results for your cloud project? Feel free to leave your thoughts in the comments section. And if you have questions on how to optimise the cost and performance of your cloud infrastructure, don’t hesitate to get in touch with our experts!

By Sergii Bielskyi, Cloud Architect at ELEKS.

13 responses to “How Azure Web Sites Sucked in Production”

  1. sestocker says:

    WordPress on Windows is almost always a bad idea. But to your point, these limitations need to be spelled out ahead of time with user warnings via email. I don’t run any any high traffic (or even moderate traffic) WordPress sites so shared plans work (I like NearlyFreeSpeech). Media Temple has a compelling WordPress plan where core updates are auto applied and there is a staging environment: http://mediatemple.net/webhosting/wordpress/

  2. nonuby says:

    I dont see a problem here. PHP limit is per process/thread (practically same thing in linux), which is generally a thread per request, with a dog like wordpress issuing serial mysql queries per request then and say each request takes 250ms to complete then just 16 req/sec would fcuk. Dont blame Azure for PHP request model sucking.

    • Yuriy Guts says:

      Agree, we should have load-tested better. But what I blame Azure for, though, is not mentioning the query quota at all, as well as not mentioning the fact that the associated ClearDB plan is free. Basically the only thing the customer sees in the documentation is the 20MB storage limit. Getting to the ClearDB portal from the Management Portal is not that obvious, unfortunately.

  3. nonuby says:

    ClearDB freemium plan of course has limited, just like Firebase, noted in the FAQ. Of course if you missed this you’d have picked it up in real world testing (with blazemeter or otherwise). 500 generally indicates a problem with your code, possibly session handling presuming frontend had sticky session which may of not been the case (hence periodic issues?). I think you’re fast to blame Azure here.

  4. damienguard says:

    WordPress is a system hog. Without W3Supercache or TotalCache it won’t handle any kind of moderate load. This is especially more noticeable on Windows as it was never optimized for IIS and barely runs. Perhaps the title should be “How Azure Web Sites Sucked in Production for WordPress” because for .NET applications they’re awesome.

  5. Ken Smith says:

    For what it’s worth, I’ve had pretty good luck with Azure websites, but I’ve always run .NET code on them. Given that probably 99% of all WP websites run on Linux, I would be surprised if running it on Windows didn’t have some significant gotchas. LAMP is supported on Azure, but it doesn’t seem to be quite the first-class citizen that .NET is.

  6. Rob Conery says:

    As Damien mentions above – you can’t use WordPress in any capacity (especially in production!) without rolling in some kind of caching. Cloudflare, TotalCache – *something*. This would have mitigated much of your trouble. Also – yeah a shared plan on a production site with 10K visits using WordPress… I might suggest reading the limits of said plans before you push live. I have been bitten as well, but the shared plan is rather stingy.

    • Yuriy Guts says:

      @Rob: I must admit that the 10K pageviews statement sounded a bit too loud. This was the kind of traffic we got because of a HN effect that day. Usually, when we don’t publish new content, the traffic is much, much lower.

  7. michelebusta says:

    We’ve been very successfully running Azure Websites for large scale customers for over 1 year now. The trick is, you have to get a proper VM, standard instance. As with any cloud deployment, database retry is a necessary evil, and understanding the usage patterns of the site so that you can ensure the appropriate VM size. I agree that the free version isn’t acceptable for most anything if you have traffic, quota shutdown sucks.

    • Yuriy Guts says:

      Yep, we use that sort of environment for our corporate website. Though we launched it when WAWS was still in preview, so it’s just a standard Web Role on a dedicated VM.

  8. ruslany says:

    Yuriy, the recommended way to run a production site on WAWS is to scale it to Basic or Standard mode which would put the site on a separate VM and will remove any quota limits. As with regards to the database outages – I had the same problem when I was migrating my WordPress-based blog onto WAWS. I eventually upgraded the database to a paid option. It looked like the free ClearDB option was only suitable for a development usage, but ran into limitations under more or less real production load.
    Thanks for the feedback on the quotas functionality for shared sites – we will look into ways to improve it.

    • Yuriy Guts says:

      Thanks for the advice Ruslan. The main motivation for using a shared plan was a modest volume of traffic we normally serve, as well as the fact that for two years our blog has been able to work on a shared hosting (I assume Blogger is) without interruptions. Looking back at all this: we made quite a few mistakes on our part, although I wish Azure documentation provided more details about ClearDB limitations (unfortunately, the pricing page doesn’t even mention the existence of ClearDB at all). This might have eliminated a lot of confusion for us as first-time ClearDB users. Anyway, we’ll be very happy to see the shared mode evolve since it’s still an awesome place to start building new projects.

  9. whyJoe says:

    We were just in the same boat. As far as all the IIS/Windows comments, or the you-should-have-read-the-limits comments… 1) I’ve run WordPress on Windows shared hosting on many other low-cost providers without these issues. I always have had Windows hosting plans. 2) Yeah, ok, the limits are spelled out somewhere. But the point is, the limits are too low, making it an uneconomic solution for a small-traffic site. Other shared hosting providers offer a much better value. On Azure, there is basically no way to do WordPress other than a VM where you can install your own MySQL instance (even though that’s less than ideal for a number of reasons.) I hit ClearDB’s low query limit even on a PAID plan, JUST with developer traffic.

Leave a Reply

Your email address will not be published. Required fields are marked *