Android Performance: Java vs Xamarin vs Xamarin.Forms Part 1

I created a short comparison of performance for Java and Xamarin. This test is only for Android, there is no comparison for iOS. I come from C# and Windows development and I’m more familiar with Xamarin. I wanted to check if Xamarin and Xamarin.Forms are now good competitors for Java.

Xamarin versions

Xamarin version: 6.1.0.71

Xamarin.Forms version: 2.3.0.49

Test 1: App size

This is size of new, empty project. There is just one Page/Activity with “Hello world” TextView/Label.

As we can see Xamarin apk size is few times bigger than native. Xamarin.Forms apk is almost 13 times bigger. All apks are in Release mode. Xamarin.Forms may be an overkill for some use cases, but I think that usually it won’t make any difference. Nowadays devices have a lot of space and media are mostly kept in the cloud, so small size is not that important. However there are still countries where Internet connection is poor or limited. If we target that countries then it can matter.

Test 2: Start time

Again, this is empty “hello world” app. I did this test in 2 ways. First method was to measure it by hand with a stopwatch. I made 10 measurements and took average.

Java is very fast here. Xamarin is about 56% slower. A bit longer starting time is still fine and not very noticeable. Xamarin.Forms is 87% slower than Xamarin and 192% slower than Java. This is already more than 2 seconds and we have to wait a bit. Not very annoying, but something that user will notice.

Second method was to measure start time with Xamarin.UITest framework. I was starting and closing app 10 times. Later I took average.

Here results are completely different. I made sure that apps are terminated before next launch, so they were not loading from memory. App was started simply with StartActivity and was marked as launched when “hello world” label was loaded. I’m not sure why there is such big difference for Xamarin. However tests made by hand reflect real world scenario better, so we should probably look at them.

Test 3: Page + ListView load time

In this test we open a new page with a button and immediately we also load a ListView with 10000 items. Item has layout simple_list_item_1, so this is just a text.

These results are very surprising! Java is fastest, this is ok. I would expect that Xamarin would have similar speed, because UI is based on the same xml files, just like in Java. But for some reason Xamarin is way slower. It is even slower than Xamarin.Forms, which uses xaml files – for this reason I expected forms to be the last one. I made this test twice – once with UITest and once by hand. Both test had analogous results. I decided to make one more similar test.

Test 4: Page + ListView + scrolling time

Here we are loading a new page and we measure the time. Then we load 10000 items into ListView and we measure time. This time list item consists of title, subtitle and an image. After loading whole ListView we scroll to the 100th item and we also measure time. During scrolling I was looking at smoothness. This test was performed by UITest framework.

Ok, that look good and reasonable so far. Xamarin is faster than Java, but they are so close that it does’t matter that much. They are pretty much the same.

That is very similar to the previous results and something I would expect. Java is about the same speed as Xamarin and Xamarin.Forms is slower. I will just note here that I was using Xamarin.Forms ListView, I was not using Custom Renderer.

I can say that scrolling for Java and Xamarin was the same – smooth, without any freezes. Xamarin.Forms on the other hand has terrible ListView. It is very painful to watch it when you scroll it. Even with such simple list it is a slide show.

This was all measured programmatically and times are a bit higher than if we did this by hand. I assume this is the case, because of how times are checked with UITest. I will also show you times for page load + button click for loading list view items + waiting till list is filled done by hand, so you can see this in real world scenario. I didn’t measure page load and list load separately because it was really fast when done by hand.

As you can see these times are much smaller than with UITest although the proportions are similar. Java was pretty much instant, Xamarin was more or less the same. With Forms we could already see a short moment when list was loading.

And for scrolling done by hand:

Numbers don’t say much here, but it was just like with UITest – Java and Xamarin were very smooth, I didn’t notice any difference. Xamarin.Forms was still very bad. I must also mention that it was this bad with Test 4 list (so title + subtitle + image). If we try scrolling with Test 3 list (one line of text) then it was pretty smooth for Forms too. However in any real world scenario this ListView is rather useless and most likely we would have to use Custom Renderer.

Very interesting here is how differently Xamarin behaves in Test 3, when we load page and list together and Test 4, when this is done separately.

Summary

As we could expect Java is the fastest. Although Xamarin is really close and end user probably wouldn’t see any difference. Unfortunately Xamarin.Forms is a different story and worse performance can be noticed.

This is the end of the first part. Next week I will try to publish next post with some more test focusing more on different operations, than UI and loading times.