Serilog ASP.NET 6 configuration

Here is the simple approach to log to console, file and database at the same time;

Step 1 — Add Environment Variable

The environment variable, ASPNETCORE_ENVIRONMENT, will be used to determine whether the application is running in Development mode or Production mode. Setting it as “Development” here allows us to put configuration values in appsettings.Development.json, which will get created soon.

Step 2 — Install Serilog NuGet packages

Assuming we want to write the logs to both the Console and also rolling text files, we can use the following Serilog packages:

Step 3— Create appsettings.Development.json

This is a JSON file where we store configuration values that will be used when the application runs in development mode, which is specified in Step 1. As you can see in the code snippet below, logging will be written into Console, Database and a rolling file with a rolling interval of day, which means each day, a new text file will be created for logging. The name of the text will become log-log-20210123.txt, while the date is appended by Serilog.

  "Serilog": {
    "Using": [
      "Serilog.Sinks.Console",
      "Serilog.Sinks.File",
      "Serilog.Sinks.MSSqlServer"
    ],
    "MinimumLevel": {
      "Default": "Debug",
      "Override": {
        "Microsoft": "Warning",
        "System": "Error"
      }
    },
    "WriteTo": [
      {
        "Name": "File",
        "Args": {
   
          "path": "C:\\Home\\LogFiles\\AppName\\log.txt",
          "outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{Level}] {Message}{NewLine}{Exception}",
          "rollingInterval": "Day"
        }
      },
      {
        "Name": "Console",
        "Args": {
          "theme": "Serilog.Sinks.SystemConsole.Themes.AnsiConsoleTheme::Code, Serilog.Sinks.Console",
          "outputTemplate": "[{Timestamp:yyyy-MM-dd HH:mm:ss.fff}] [{Level}] {MachineName} ({ThreadId}) <{SourceContext}> {Message}{NewLine}{Exception}"
        }
      },
      {
        "Name": "MSSqlServer",
        "Args": {
          "connectionString": "Data Source=url;Initial Catalog=FOO;Persist Security Info=True;User ID=Adam;Password=xyz",
          "sinkOptionsSection": {
            "tableName": "MyLog",
            "schemaName": "dbo",
            "autoCreateSqlTable": true,
            "batchPostingLimit": 1000,
            "period": "0.00:00:30"
          },
          "columnOptionsSection": {
            "disableTriggers": true,
            "PrimaryKeyColumnName": "Id",
            "addStandardColumns": [ "LogEvent" ],
            "removeStandardColumns": [ "MessageTemplate", "Properties" ],
            "timeStamp": {
              "columnName": "Timestamp",
              "convertToUtc": false
            }
          }
        }
      }
    ]
  }

You can clear the appsettings.json content if all configuration values are stored in environment-specific json file, and just leave an empty curly brackets, {}.

Step 4 — Configure Serilog in Program.cs

Here is how;

builder.Host.UseSerilog((context, provider, config) => {
    config.ReadFrom.Configuration(context.Configuration);
    });

app.UseSerilogRequestLogging();

Resources

A reference from Code Maze web site

Here is a good reference

Azure App services debugging and monitoring

It is very familiar. “XXX works fine locally but does not work at all in production”.

I am going to take assumption that the environment variable, ASPNETCORE_ENVIRONMENT, will be used to determine whether the application is running in Development mode or Production mode. Setting it as “Development.

Next move to Azure App services.

Within Azure Portal, add ASPNETCORE_ENVIRONMENT to the Application settings. Instead of “Development”, we will use “Production” here. Navigate to the App Service for the application, then navigate to Configuration:

Turn on App Service Logs to Filesystem in App Service

Within Azure Portal, navigate to App Service logs, Turn on Web server logging. This creates the folder path “D:\home\LogFiles\http\RawLogs” and starts writing server log files there.

A few benefits of us writing Serilog rolling files in the same location:

1-One single spot to access logs

2-When Application Logging (Filesystem) is turned on, you can view the Serilog logs in the Log Stream under App Service Monitoring section as well! Awesome!

Test and view the log files

Assuming your app service URL is https://YourWebAppName.azurewebsites.net, while being logged into Azure Portal, visit https://YourWebAppName.SCM.azurewebsites.net which is the Kudu site. You should be able to locate the log files by going to Debug console/CMD and navigate to “D:\home\LogFiles\http\RawLogs”.

Another spot to diagnose the problem is to look here;

And on the next page, look here;

It is easy to develop and test locally but when you deploy the application to Azure App Service, a few extra steps are required so that logs are properly populated and accessible within the Azure App Service.

Resources

https://shawn-shi.medium.com/proper-use-of-serilog-for-log-stream-and-filesystem-on-azure-app-service-a69e17e54b7b

Ajax Request: Response to preflight request doesn’t pass access control check

Today, I started getting this error after making few Ajax calls to a remote server;

Access to XMLHttpRequest at 'https://foo.com' from origin 'http://localhost:3000' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.

This problem relates to CORS. Here is some explanation and work around;

https://stackoverflow.com/questions/35588699/response-to-preflight-request-doesnt-pass-access-control-check

Here is some more useful info;

https://www.edureka.co/community/82342/how-to-add-custom-http-header-to-ajax-request-with-javascript

JavaScript Callback function

In JavaScript, we can pass a function to another function as an argument. By definition, a callback is a function that we pass into another function as an argument for executing later.

We are going to focus on Asynchronous callbacks. An asynchronous callback is executed after the execution of the high-order function that uses the callback.

Asynchronicity means that if JavaScript has to wait for an operation to complete, it will execute the rest of the code while waiting.

Here is an example of Asynchronous callback;

Suppose that you need to develop a script that downloads a picture from a remote server and process it after the download completes:

function download(url) {
    // ...
}

function process(picture) {
    // ...
}

download(url);
process(picture);

However, downloading a picture from a remote server takes time depending on the network speed and the size of the picture.

The following download() function uses the setTimeout() function to simulate the network request:

function download(url) {
    setTimeout(() => {
        // script to download the picture here
        console.log(`Downloading ${url} ...`);
    },1000);
}

And this code emulates the process() function:

function process(picture) {
    console.log(`Processing ${picture}`);
}

When we execute the following code:

let url = 'https://www.foot.net/pic.jpg';

download(url);
process(url);

we will get the following output:

Processing https://foo.net/pic.jpg
Downloading https://foo.net/pic.jpg ...

This is not what we expected because the process() function executes before the download() function. The correct sequence should be:

  • Download the picture and wait for the download completes.
  • Process the picture.

To resolve this issue, we can pass the process() function to the download() function and execute the process() function inside the download() function once the download completes, like this:

function download(url, callback) {
    setTimeout(() => {
        // script to download the picture here
        console.log(`Downloading ${url} ...`);
        
        // process the picture once it is completed
        callback(url);
    }, 1000);
}

function process(picture) {
    console.log(`Processing ${picture}`);
}

let url = 'https://wwww.javascripttutorial.net/pic.jpg';
download(url, process);

Output:

Downloading https://www.foo.net/pic.jpg ...
Processing https://www.foo.net/pic.jpg

Now, it works as expected.

In this example, the process() is a callback passed into an asynchronous function.

When we use a callback to continue code execution after an asynchronous operation, the callback is called an asynchronous callback.

To make the code more concise, we can define the process() function as an anonymous function:

function download(url, callback) {
    setTimeout(() => {
        // script to download the picture here
        console.log(`Downloading ${url} ...`);
        // process the picture once it is completed
        callback(url);

    }, 1000);
}

let url = 'https://www.javascripttutorial.net/pic.jpg';
download(url, function(picture) {
    console.log(`Processing ${picture}`);
}); 

Handling errors

The download() function assumes that everything works fine and does not consider any exceptions. The following code introduces two callbacks: success and failure to handle the success and failure cases respectively:

function download(url, success, failure) {
  setTimeout(() => {
    console.log(`Downloading the picture from ${url} ...`);
    !url ? failure(url) : success(url);
  }, 1000);
}

download(
  '',
  (url) => console.log(`Processing the picture ${url}`),
  (url) => console.log(`The '${url}' is not valid`)
);

Nesting callbacks and the Pyramid of Doom

How do we download three pictures and process them sequentially? A typical approach is to call the download() function inside the callback function, like this:

function download(url, callback) {
  setTimeout(() => {
    console.log(`Downloading ${url} ...`);
    callback(url);
  }, 1000);
}

const url1 = 'https://www.foo.net/pic1.jpg';
const url2 = 'https://www.foo.net/pic2.jpg';
const url3 = 'https://www.foo.net/pic3.jpg';

download(url1, function (url) {
  console.log(`Processing ${url}`);
  download(url2, function (url) {
    console.log(`Processing ${url}`);
    download(url3, function (url) {
      console.log(`Processing ${url}`);
    });
  });
});

Output:

Downloading https://www.foo.net/pic1.jpg ...
Processing https://www.foo.net/pic1.jpg
Downloading https://www.foo.net/pic2.jpg ...
Processing https://www.foo.net/pic2.jpg
Downloading https://www.foo.net/pic3.jpg ...
Processing https://www.foo.net/pic3.jpg

The script works perfectly fine.

However, this callback strategy does not scale well when the complexity grows significantly.

Nesting many asynchronous functions inside callbacks is known as the pyramid of doom or the callback hell:

asyncFunction(function(){
    asyncFunction(function(){
        asyncFunction(function(){
            asyncFunction(function(){
                asyncFunction(function(){
                    ....
                });
            });
        });
    });
});

To avoid the pyramid of doom, we use promises or async / await functions.

How single page application works

Single page applications use hash-based or pushState navigation to support back/forward gestures and bookmarking. The best known example probably is Gmail but now there are other too. If you’re not familiar with how that technique works, here is a brief explanation;

For hash-based navigation, the visitor’s position in a virtual navigation space is stored in the URL hash, which is the part of the URL after a ‘hash’ symbol (e.g., /my/app/#category=shoes&page=4). Whenever the URL hash changes, the browser doesn’t issue an HTTP request to fetch a new page; instead it merely adds the new URL to its back/forward history list and exposes the updated URL hash to scripts running in the page. The script notices the new URL hash and dynamically updates the UI to display the corresponding item (e.g., page 4 of the “shoes” category).

This makes it possible to support back/forward button navigation in a single page application (e.g., pressing ‘back’ moves to the previous URL hash), and effectively makes virtual locations bookmarkable and shareable.

pushState is an HTML5 API that offers a different way to change the current URL, and thereby insert new back/forward history entries, without triggering a page load. This differs from hash-based navigation in that you’re not limited to updating the hash fragment — you can update the entire URL.