For Enabling Anonymous Access for custom application Page try this,
Step 1: Open the Sharepoint Central Administration
2: Click Manage Web Application(your custom web application)
->Authentication Provider->open Default->Enable Anonymous
Step 3: Goto to the Sharepoint Custom Site -> Site Permissions -> on the ribbon click Anonymous Access .
Step 4: choose whether you need 1. Entire website 2. list and libraries.