Combining data+notification payload in FCM Android

If you send this payload to an iOS device via FCM, and also invoke UNUserNotificationCenter's didReceive response: withCompletionHandler callback method, iOS will both display a text notification as well as allow you app some background time to deal with the data object you're sending along your push notification.

Unfortunately on Android even using my recommended payload you won't achieve this behavior. If you send both notification and data, only the notification will be shown, and the OnMessageReceived callback method won't be called to handle the data object.

But if you're in charge of the FCM payload creation, you can put the notification's title and body into the data object like this:

{  
   "message":{  
      "token":"<device registration id>",
      "data":{
         "account":{
            "first-name":"Igor",
            "last-name":"Z"
         },
         "androidTitle":"title",
         "androidBody":"body"
      }
   }
}

And when OnMessageReceived is called with the contents of this payload, you just create a local heads-up notification based on the info in androidTitle and androidBody with this code:

This code is a bit verbose, but is guaranteed to be working on all current versions of Android, which is invaluable 🙂 You just put androidTitle and androidBody as parameters of the method above and a nice default nofitication will appear on each message.

Now you can enjoy notification+data push notifications in a single message on Android with this hack, instead of sending two messages each time.

Firebase Android notification payload

Unlike iOS, for Android Firebase's FCM server accepts payload in slightly different form:

{  
   "message":{  
      "token":"<device registration id>",
      "android":{  
         "notification":{  
            "title":"title",
            "sound":"default",
            "body":"body"
         },
         "priority":"high"
      },
      "data":{
         "account":{
            "first-name":"Igor",
            "last-name":"Z"
         }
      }
   }
}

But there's a caveat: when both the android and data objects are present - only the notification is shown, without data being passed to OnMessageReceived FCM callback method. So in order to send both data and notification at the same time on Android you either have to send two messages - one data and one notification. Or to be a bit more creative 🙂

Firebase iOS push payload

If you're using Firebase and its FCM (Firebase Cloud Messaging) server directly to send out notifications to iOS users, you might wonder what kind of payload you should send to Firebase in order for the information to be delivered.

So wonder no more 🙂

{
   "message":{
      "token":"<device registration id>",
      "apns":{
         "payload":{
            "aps":{
               "content-available":1,
               "alert":{
                  "title":"title",
                  "subtitle":"subtitle",
                  "body":"body"
               },
               "badge":7,
               "sound":"default"
            }
         }
      },
      "data":{
         "account":{
            "first-name":"Igor",
            "last-name":"Z"
         }
      }
   }
}

In case you need, here's an example of the payload for Android

E-tickets

Few weeks ago I was in a rush and after coming to the railway station I had a choice - either stand in line to buy a ticket for the train but miss it, or get on it without one and risk getting a fine.

E-tickets

Of course both options were bad, but the downside of being late was worse than paying a fine, so I jumped on the train, but still managed to get the ticket without standing in line to the ticket machine.

After getting on the train I realized I had an app installed that allowed buying train tickets online, and it even had few credits left from my previous top-up. So I used it and spent the rest of the trip not worrying about fines.

The moral of the story is that I highly recommend getting to know online ticket services in your area - it simply saves you time from standing in lines. And saves paper 👍 If of course you're using public transportation 🙂 If not - I'm sure there are other areas covered with online reservations where you can improve your experience, avoid lines or just save time.

Setting up your own DNS server with Pi-Hole

While looking through my custom DNS server settings you might have noticed I actually have 1.1.1.1 as my secondary DNS server with my local 192.168.x.x being first. I went a step further of setting up pi-hole's DNS server in my local network and am using it as my primary DNS server when at home.

DNS

Pi-Hole has stricter filtering rules than of the more neutral CloudFare's DNS server which in our day and age can't be too strict. Also since I'm in control of this DNS server I can trust it completely, unlike other public pi-hole servers which might be secure but also might be not. In case you have the option to setup your local DNS server - I highly recommend doing it, especially since pi-hole makes it easier to do, you'll just need a local server or even a VM to run it on. If you're using only one device, technically you can run it even off your machine and connect to it there as well, but having a separate solution is still nicer, especially so you can connect other device to your local ads filtering DNS server, and have faster and safer Internet in all the apps and services you use on all of your devices.

But what if you are browsing the web not from your home and still want to use all of the advantages of custom DNS servers, especially on iOS when there is no way of overriding your carrier's DNS? On Android using a local filtering VPN service is your best bet. There are few options for your desktop OS and only on iOS apparently the single solution is to use what is called DNSCrypt. But at least the option count on iOS is greater than zero 🙂

Custom DNS on iOS via DNSCrypt

DNSCrypt

In previous posts about adblockers and VPNs for iOS I covered all the pros and cons of both approaches. TLDR: on iOS adblockers help you only in the browser but barely improve your privacy, whereas VPNs do both well but at the expense of your Internet speed, both on cellular and Wi-Fi.

After going through blockers, I mentioned setting custom DNS servers as a mean of filtrating ads and trackers on the domain name level. And you can do that fairly easily on Mac, iOS and Android while being connected to the Internet via Wi-Fi but you can't do that on cellular.

...Actually there is a chance and it's called DNSCrypt. In short this is a way of communicating with a DNS server not via regular DNS protocols which your carrier and your mobile device don't let you adjust. The connection to a DNS server of your choice is established via HTTPS, so it's secure, and you can customize it. The only requirement is that your DNS provider of choice should support resolving domains via HTTPS (usually 443 port) in addition to the usual 53 DNS port.

Luckily, my 1.1.1.1 DNS server of choice (provided by CloudFlare) supports DNS queries via HTTPS as well. So the only thing I had to do is to install DNSCloak for iOS, find 1.1.1.1 in the supplied list of DNSCrypt-enabled DNS servers and push 'start'. That establishes a 'VPN' connection which is not actually VPN since it doesn't send all your traffic to another server, just the DNS queries. And as a result you get your ads and trackers filtered on a domain level, without the downside of speed decrease which all traditional VPNs have in common.

As a recommendation you can make your DNSCrypt connection to be more stable. To do that open Settings.app, then go to General -> VPN, tap the 'i' button next to 'DNSCloak' and then switch on 'Connect On Demand' at the bottom.

In case you have your own pi-hole DNS server facing the open Internet (which you should do carefully or don't do at all), or you know a public one you can trust - you can enable it in DNSCloak and have even more strict DNS filtering than Google or CloudFare provides. They are public DNS servers and they are more conservative on filtering out stuff not to accidentally block websites used by the general public (e.g. blocking Facebook's tracking 'like' buttons may block Facebook at all). But that doesn't mean your pi-hole DNS server of choice can't be more strict 🙂

VPN (mobile and desktop): the good and the bad

If you want the best privacy protection as well as great ad tracking prevention on iOS, VPNs are your best bet, although without some caveats.

VPN-on-iOS

First off, let's start with what a VPN means. VPN is a Virtual Private Network which means connecting your remote devices into a single private but virtual network since the devices may not actually be connected directly to each other.

Usually when your device makes a connection to any service, the data you're sending and retrieving travels through dozens of nodes until it reaches the server on the other end. And you can't be sure none of the nodes in between aren't compromised and for example aren't storing your data without your consent. The main advantage of using VPNs is that all of your Internet traffic travels encrypted and is available only to you and the server and no one in between. Of course it's not a bullet-proof solution but it's the best and easiest way to make your data as much unusable as possible to non authorized middleman parties. Especially if you're controlling your VPN, since paid or public VPNs which provide transparency and security might not actually do it and do the exact opposite - log all of your data for their own benefit.

For most of the people, and even for advanced users using a VPN might be overwhelming and hard, especially if you decided to set it up yourself and not to use a public one. And without specific knowledge such setup may be leaking your data even more than no VPN at all. But the manuals like this cover the security basics quite well, as well as some of the public VPN providers hold many years of trust without being revealed in scandals of (un)intentional data leaks.

So using a VPN generally is a good idea - you get your traffic encrypted which keeps your private data safe and as a side effect you get even some ad tracking prevention since even your basic data stops being available to the advertisers who keep their trackers on many websites and services you visit. And if you're using your device to access sensitive data by using public wifi or in countries with regulated Internet access - VPNs are a must for you. Also if you want to access country specific/restricted resources, there is not much choice except using a VPN at all, which in this case would make you have an external IP address that belongs to that country and you will appear like you're browsing from within that region - that's another advantage of using VPNs.

If you plan on using VPNs, secure the most important devices first. But if you can - use VPNs on all of your devices. I tried going this route, but here are some downsides that made me stop using them:

  • Using VPNs on any of the device will make the Internet feel slower. The reason for it is that instead of your packets going directly to the server you're trying to reach they go a longer route through your secure server first, and that adds up delay to each of your web request and as a result everything loads noticeably slower.
  • Also your max speeds might take a hit. With my 250/20 Mbit speeds at home I was getting about 20/10 Mbit using a public VPN since they are often sharing their channels within many users. Setting up your own VPN somewhere with high speed Internet access should make it better, but the delays mentioned before still won't improve much and will also cripple your speeds.
  • The speed hit affects mobile devices connected via cellular the most since those already have a pretty high delay of transferring data through the air. So on mobile the speed slowdown is noticeable the most.
  • The other problem (at least with iOS) is that the device might disconnect from the VPN in the background and leave portions of your background traffic unsecured as well as some traffic before it reconnects after unlocking the phone. From what I understand the VPN disconnects to preserve battery which it uses 10-15% more of than without having an active VPN connection - this in my opinion is also a significant problem with VPNs on the go.

That said, if you need your connections to be secure and untraceable - you still have the option even though it would cost you Internet speed and some battery life, but if you need it in certain circumstations, that's a fair tradeoff.

In my tests I was using Tunnelbear (loved its cuteness and service, until it was purchased with a not very trust McAffee recently), SurfEasy and my new favorite PIA because of their high long standing reputation. And you should care about the reputation of the company that keeps your traffic private 🙂 As for setting up your own VPN - I used DigitalOcean's manual from the link above but with Ubuntu 16.04 instead of 18.04.

If you don't need that level of security and just want to browse the Internet without ads, here's my breakdown on the best iOS and Android adblockers as well as a post on what privacy concerns you should consider while using and adblocker in the first place.

But if you want something better for filtering ads outside of Mobile Safari on iOS and on other devices, you can switch your DNS server on them to have better filtration all the time without any penalty on the battery. And if you want to have custom DNS servers for your iOS device not only in your local Wi-Fi area, you can setup DNSCrypt and have ads filtering on the DNS level everywhere you go!

Custom DNS Servers

In my summary of iOS adblockers I brought up a topic of using custom DNS servers as a better way of limiting ads and tracking and here's why. On iOS this method allows to limit them even outside Safari, which is very good news, since there are no actually working ways of keeping other apps and services from tracking you unless you jailbreak your device.

DNS

I'll be getting into details of setting custom DNS Servers and DNSCrypt on iOS since it's the most limited platform in terms of options. All of the steps below are available on Windows, Mac and Android as well, which makes it the most versatile option for the most devices possible.

So what is a DNS? DNS is abbreviated from Domain Name System - it's a special network of servers which purpose is just to resolve domains. What that means is that when you enter any address in your browser, or any app or service that you're using connects to its service via a domain, your device which allows that connection connects to its DNS server and asks for the IP address of the domain you're connecting to. If you open google.com, your computer or mobile phone asks the IP address of the domain, receives 216.58.215.78 and connects your browser to it. But if for some reason the DNS server doesn't resolve your DNS request, your browser won't know where to connect and as a result you won't see anything. This is exactly what can be used not to see ads.

DNS is a hierarchical network with few root servers at the top. The system is built to be fail safe with backup servers in each node, but from its nature there were times when a portion of them went down making parts of the Internet inaccessible to some users. For that reason ISPs (Internet service providers) have their own copies of the main DNS servers in case they might temporary fail. They keep those copies up to date with the main servers so when there's an outage the IPSs' users won't feel anything since their DNS queries will still resolve on the ISP servers. Also potentially resolving your DNS queries in place speeds up your loading speeds comparing to the situation when your DNS query would go further than your ISP servers which would just take longer to do.

Usually when you connect your device to any router, it will give you static IP addresses of the DNS servers the router is set up to share. The routers of your ISP are pre-set with their DNS servers, the same happens when your phone gets cellular connection - phone's DNS servers are also set for you. And most ISPs in order to keep users away from messing up with their Internet keep those DNS IP addresses hidden from change.

But why would you want to change your DNS servers? Well there are few reasons for that:

  1. Your DNS provider (most often your ISP) has his own non-objective interests in mind. If it's a governmental ISP in Russia, they might wanna block their subjectively harmful websites, and not resolving the DNS query for those domains is an easy way of doing it. Your browser would just tell you that it 'couldn't resolve host name'. Or if your ISP provides additional services, they might just block their online competitors that way, leaving you out only with the options they want you to have. These examples are a bit extreme, but worth mentioning since all of the scenarios are easily possible. Even though when blocking, ISPs do a bit further and just block all the traffic to specific resources and not just DNS queries. And in that case only VPNs are your only option to get around the limitations.
  2. Your DNS provider has less incentive than you on blocking ads and trackers since it sometimes might break the website you're watching or maybe your ISP shareholders are also partial owners of an advertising group, etc. And this is where your custom DNS server might help you out when your ISP won't, by setting up few rules not to resolve the domains which usually serve ads or track your online behavior and identity and afterwards sell that information again to advertisers.

As mentioned before the DNS system consists of root and other big servers that resolve people's queries from all over the world. But not only your ISP has a copy of the domain names and their IP addresses. Any private company or even regular people can set up their own DNS server. The only difference is that the root servers are operating mostly independently, following objective rules and are considered safe to be used by everyone, unlike some Joe's public DNS server. Big companies also have their own DNS servers, in particular Google with its 8.8.8.8 and CloudFare (cloud service provider) with 1.1.1.1. And even though Google is a trusted company, I wouldn't trust them to resolve my DNS queries, even though they probably filter out malicious ones for good reasons, but their core business is about ads, so they may be doing that for competitive reasons, filtering competition out, and leaving only their own domains responsible for tracking 🙂

That's why after CloudFare revealed their DNS few months ago I started using them right away, since their core business relies on making the Internet faster and safer not as a byproduct of serving ads. I couldn't switch my router to serve their DNS to all my home devices since my ISP hid the option, so I had to setup all my devices manually.

DNS-Mac

It's very easy doing it on Mac OS: you go to Settings.app -> Network, push 'Advanced' on your connection of choice (Wi-Fi, Ethernet, etc.) and on the 'DNS' tab add 1.1.1.1 and confirm the changes - that easy! You can repeat it on all of your Macs and PCs and this setting will stay even while connecting to other wireless or wired networks, neat!

On Android and iOS the DNS setting is per each wireless network. So if you use few on a constant basis - unfortunately you will have to repeat this process for each of the networks: open Settings.app -> Wi-Fi, tap on the 'i' icon next to your connected network and then on 'Configure DNS' at the bottom, select 'Manual' and enter 1.1.1.1 as well. Save changes, enjoy your faster and more secure Internet 🙂

DNS-iOS

The setup on Android is almost identical and I'm sure you will find it after going through the same sequence.

iOS adblocking options (August 2018)

After getting into a deep dive with Android adblockers I started wondering whether anything changed on iOS (my daily platform of choice) since the last time I researched this topic.

Adblock-ios

iOS adblockers were first approached by Apple with the release of iOS 9 when they introduced an adblocking API for Mobile Safari. The API was so simple and limiting at the same time that few developers pushed out a couple of adblockers literally within days after the iOS 9 worldwide rollout. And the reason was that anyone could build an adblocker for iOS in a matter of few days - it was (and is) that simple. The differentiating part is just the list of hosts (filters) to block was slightly different between the adblockers but the main idea was the same: the user had to open Settings.app -> Safari -> Content blockers and enable his adblocker of choice, thus allowing the system block the hosts included in that app. So in order to launch an adblocking app you could just scrape few filters, maybe some opensource ones and you're done 🙂

The problem was with the limited amount of filters per one app. Even though 50 000 hosts sounds like a lot, in practice it wasn't enough to block ads efficiently. I started my journey of finding the best iOS adblocker right after the new iOS release with Purify, then with Peace (which was live only for two days), Crystal and ended up using 1Blocker (which is now 'Legacy'). I decided on 1Blocker since it has a Mac app with iCloud sync of your custom filters, whitelists, settings etc.

After a year of using 1Blocker I started noticing more and more ads in Safari. It felt like the developer abandoned the app which later was proven by him releasing 1Blocker X - an updated version of the adblocker the developer supposedly was working on. Not willing to support the developer's new version which might be abandoned someday with his next work, I started searching Reddit in order to find a replacement. Some people suggested using AdGuard, others replied not to trust an adblocker made in Russia with your browsing info which made sense. People were recommending many adblockers I already tried, including 1Blocker, mostly because of its strong word of mouth. Then in the end I saw few comments recommending Wipr and seeing it's 4.8 star rating helped me to decide in its favor.

Adblock-ios2

Wipr goes around the limitation of 50k hosts by setting itself up as three content blockers all of which you should enable in the Settings.app which is quite clever. But the most important thing besides the app's clean simple interface is that it actually does the job - now I'm seeing less ads, pages are loading fast again.

Even though I now had a new working adblocker I was still unsatisfied. I couldn't believe that having only an adblocker only in Safari is the only option in fighting ads and tracking on iOS. But then I recalled that in addition to an adblocker on macOS I'm using custom DNS servers, two of them to be exact.

So if you want basic working adblocking on iOS - you now have few choices. As for additional protection you can read my followup on custom DNS servers on iOS and in general.

And if you're into complete and best possible privacy options, you can read my take on using VPNs on your devices with some specifics about iOS.

Facebook's misleading ads

I was scrolling my timeline in Facebook's iOS app the other day when I got this bizarre ad:

Facebook-ads1

As a drone owner myself I actually recognized my DJI Mavic Air in the video. And what got my attention was the episode where some guy threw a water balloon at the drone. That's where I first thought it's an ad for some sort of water protection glue or skin which allows you to make your drone resist water since most of them are afraid of it.

I naturally clicked the ad since it resonated well with me (praise Facebook's targeting!) and the idea I can preserve my drone from water damage. But when the ad's landing page finished loading I saw they are not selling any protection but the drone itself! The images again displayed the Mavic Air with the initial price of twice as much as you would pay for a new one and a discount of 95% of that price or about 88% discount of the Air's real price! And of course at the end of the page you would find a countdown urging you to purchase.

Facebook-ads2

Facebook claims they are reviewing each ad. But in the next few days I was getting all sorts of gadget ads displayed in my timeline and every time with a huge discount. And each of the ads had tons of commends with the word 'scam' in them. So the reach of those ads was big and such ad should have gotten even more Facebook's attention but it didn't. I assume the clicks pay off nicely for both the social network ad revenue and for the scammer as well.