Working with different type of WebClient to consume Web API

This is a list of web clients used in different environments to consume Web API.

Web Client in .NET Framework

To begin with, you have three different choices for consuming REST APIs when working in the .NET Framework: WebClient, HttpClient, and HttpWebRequest. In this post we will look at these three ways we can access REST APIs from within the managed environment, i.e., without resorting to third-party libraries. In the sections that follow I will illustrate these approaches with relevant code examples to help you gain a better understanding of the concepts.

In a nutshell, WebRequest—in its HTTP-specific implementation, HttpWebRequest—represents the original way to consume HTTP requests in .NET Framework. WebClient provides a simple but limited wrapper around HttpWebRequest. And HttpClient is the new and improved way of doing HTTP requests and posts, having arrived with .NET Framework 4.5.

Let’s start our discussion with the WebRequest abstract class.

System.Net.WebRequest

The System.Net.WebRequest class is an abstract class. Thus you will need to create a HttpWebRequest or FileWebRequest to consume HTTP requests using this class. The following code snippet shows how you can work with WebRequest.

WebRequest webRequest = WebRequest.Create(uri);
webRequest.Credentials = CredentialCache.DefaultCredentials;
webRequest.Method ="GET";
HttpWebResponse webResponse = (HttpWebResponse)webRequest.GetResponse();
System.Net.HttpWebRequest

WebRequest was the first class provided in the .NET Framework to consume HTTP requests. It gives you a lot of flexibility in handling each and every aspect of the request and response objects, without blocking the user interface thread. You can use this class to access and work with headers, cookies, protocols, and timeouts when working with HTTP. The following code snippet illustrates how HttpWebRequest can be used.

HttpWebRequest http = HttpWebRequest)WebRequest.Create(“http://localhost:8900/api/default”);
WebResponse response = http.GetResponse();
MemoryStream memoryStream = response.GetResponseStream();
StreamReader streamReader = new StreamReader(memoryStream);
string data = streamReader.ReadToEnd();

You can find Microsoft’s documentation on HttpWebRequest here

System.Net.WebClient

The System.Net.WebClient class in .NET provides a high-level abstraction on top of HttpWebRequest. WebClient is just a wrapper around HttpWebRequest, so uses HttpWebRequest internally. Thus WebClient is a bit slow compared to HttpWebRequest, but requires you to write much less code. You can use WebClient for simple ways to connect to and work with HTTP services. It is generally a better choice than HttpWebRequest unless you need to leverage the additional features that HttpWebRequest provides. The following code snippet shows how you can work with WebClient.

string data = null;
using (var webClient = new WebClient())
{
    data = webClient.DownloadString(url);
}
System.Net.Http.HttpClient

HttpClient was introduced in .NET Framework 4.5. For developers using .NET 4.5 or later, it is the preferred way to consume HTTP requests unless you have a specific reason not to use it. In essence, HttpClient combines the flexibility of HttpWebRequest and the simplicity of WebClient, giving you the best of both the worlds.

The HttpWebRequest class provides a lot of control over the request/response object. However, you should be aware that HttpClient was never designed to be a replacement for WebClient. You should use HttpWebRequest instead of HttpClient whenever you need the additional features that HttpWebRequest provides. Further, unlike WebClient, HttpClient lacks support for progress reporting and custom URI schemes. 

Although HttpClient doesn’t support FTP, mocking and testing HttpClient is easier. All I/O bound methods in HttpClient are asynchronous, and you can use the same HttpClient instance to make concurrent requests as well. The following code snippet illustrates how you can work with HttpClient.

public async Task<Author> GetAuthorsAsync(string uri)
{
    Author author = null;
    HttpResponseMessage response = await client.GetAsync(uri);
    if (response.IsSuccessStatusCode)
    {
        author = await response.Content.ReadAsAsync<Author>();
    }
    return author;
}

Note that when there is an error in the response, HttpClient doesn’t throw an error. Rather, it sets the IsSuccessStatusCode property to false. If you want to throw an exception if the IsSuccessStatusCode property is false, you can make a call to the EnsureSuccessStatusCode method on the response instance as shown below.

response.EnsureSuccessStatusCode();

HttpClient was designed to be instantiated once and reused throughout the application’s lifecycle—you should not create a new HttpClient instance for every request that your application needs to process. If you do, the available sockets could become exhausted by heavy traffic, resulting in SocketException errors. The recommended practice is to create a single, shared HttpClient instance.

HttpClientFactory

To do…

Web Client in Angular

To do..

Serilog ASP.NET 6 configuration

Here is the simple approach to log to console, file and database at the same time;

Step 1 — Add Environment Variable

The environment variable, ASPNETCORE_ENVIRONMENT, will be used to determine whether the application is running in Development mode or Production mode. Setting it as “Development” here allows us to put configuration values in appsettings.Development.json, which will get created soon.

Step 2 — Install Serilog NuGet packages

Assuming we want to write the logs to both the Console and also rolling text files, we can use the following Serilog packages:

Step 3— Create appsettings.Development.json

This is a JSON file where we store configuration values that will be used when the application runs in development mode, which is specified in Step 1. As you can see in the code snippet below, logging will be written into Console, Database and a rolling file with a rolling interval of day, which means each day, a new text file will be created for logging. The name of the text will become log-log-20210123.txt, while the date is appended by Serilog.

  "Serilog": {
    "Using": [
      "Serilog.Sinks.Console",
      "Serilog.Sinks.File",
      "Serilog.Sinks.MSSqlServer"
    ],
    "MinimumLevel": {
      "Default": "Debug",
      "Override": {
        "Microsoft": "Warning",
        "System": "Error"
      }
    },
    "WriteTo": [
      {
        "Name": "File",
        "Args": {
   
          "path": "C:\\Home\\LogFiles\\AppName\\log.txt",
          "outputTemplate": "{Timestamp:yyyy-MM-dd HH:mm:ss.fff zzz} [{Level}] {Message}{NewLine}{Exception}",
          "rollingInterval": "Day"
        }
      },
      {
        "Name": "Console",
        "Args": {
          "theme": "Serilog.Sinks.SystemConsole.Themes.AnsiConsoleTheme::Code, Serilog.Sinks.Console",
          "outputTemplate": "[{Timestamp:yyyy-MM-dd HH:mm:ss.fff}] [{Level}] {MachineName} ({ThreadId}) <{SourceContext}> {Message}{NewLine}{Exception}"
        }
      },
      {
        "Name": "MSSqlServer",
        "Args": {
          "connectionString": "Data Source=url;Initial Catalog=FOO;Persist Security Info=True;User ID=Adam;Password=xyz",
          "sinkOptionsSection": {
            "tableName": "MyLog",
            "schemaName": "dbo",
            "autoCreateSqlTable": true,
            "batchPostingLimit": 1000,
            "period": "0.00:00:30"
          },
          "columnOptionsSection": {
            "disableTriggers": true,
            "PrimaryKeyColumnName": "Id",
            "addStandardColumns": [ "LogEvent" ],
            "removeStandardColumns": [ "MessageTemplate", "Properties" ],
            "timeStamp": {
              "columnName": "Timestamp",
              "convertToUtc": false
            }
          }
        }
      }
    ]
  }

You can clear the appsettings.json content if all configuration values are stored in environment-specific json file, and just leave an empty curly brackets, {}.

Step 4 — Configure Serilog in Program.cs

Here is how;

builder.Host.UseSerilog((context, provider, config) => {
    config.ReadFrom.Configuration(context.Configuration);
    });

app.UseSerilogRequestLogging();

Resources

A reference from Code Maze web site

Here is a good reference

Azure App services debugging and monitoring

It is very familiar. “XXX works fine locally but does not work at all in production”.

I am going to take assumption that the environment variable, ASPNETCORE_ENVIRONMENT, will be used to determine whether the application is running in Development mode or Production mode. Setting it as “Development.

Next move to Azure App services.

Within Azure Portal, add ASPNETCORE_ENVIRONMENT to the Application settings. Instead of “Development”, we will use “Production” here. Navigate to the App Service for the application, then navigate to Configuration:

Turn on App Service Logs to Filesystem in App Service

Within Azure Portal, navigate to App Service logs, Turn on Web server logging. This creates the folder path “D:\home\LogFiles\http\RawLogs” and starts writing server log files there.

A few benefits of us writing Serilog rolling files in the same location:

1-One single spot to access logs

2-When Application Logging (Filesystem) is turned on, you can view the Serilog logs in the Log Stream under App Service Monitoring section as well! Awesome!

Test and view the log files

Assuming your app service URL is https://YourWebAppName.azurewebsites.net, while being logged into Azure Portal, visit https://YourWebAppName.SCM.azurewebsites.net which is the Kudu site. You should be able to locate the log files by going to Debug console/CMD and navigate to “D:\home\LogFiles\http\RawLogs”.

Another spot to diagnose the problem is to look here;

And on the next page, look here;

It is easy to develop and test locally but when you deploy the application to Azure App Service, a few extra steps are required so that logs are properly populated and accessible within the Azure App Service.

Resources

https://shawn-shi.medium.com/proper-use-of-serilog-for-log-stream-and-filesystem-on-azure-app-service-a69e17e54b7b

Ajax Request: Response to preflight request doesn’t pass access control check

Today, I started getting this error after making few Ajax calls to a remote server;

Access to XMLHttpRequest at 'https://foo.com' from origin 'http://localhost:3000' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource.

This problem relates to CORS. Here is some explanation and work around;

https://stackoverflow.com/questions/35588699/response-to-preflight-request-doesnt-pass-access-control-check

Here is some more useful info;

https://www.edureka.co/community/82342/how-to-add-custom-http-header-to-ajax-request-with-javascript

JavaScript Callback function

In JavaScript, we can pass a function to another function as an argument. By definition, a callback is a function that we pass into another function as an argument for executing later.

We are going to focus on Asynchronous callbacks. An asynchronous callback is executed after the execution of the high-order function that uses the callback.

Asynchronicity means that if JavaScript has to wait for an operation to complete, it will execute the rest of the code while waiting.

Here is an example of Asynchronous callback;

Suppose that you need to develop a script that downloads a picture from a remote server and process it after the download completes:

function download(url) {
    // ...
}

function process(picture) {
    // ...
}

download(url);
process(picture);

However, downloading a picture from a remote server takes time depending on the network speed and the size of the picture.

The following download() function uses the setTimeout() function to simulate the network request:

function download(url) {
    setTimeout(() => {
        // script to download the picture here
        console.log(`Downloading ${url} ...`);
    },1000);
}

And this code emulates the process() function:

function process(picture) {
    console.log(`Processing ${picture}`);
}

When we execute the following code:

let url = 'https://www.foot.net/pic.jpg';

download(url);
process(url);

we will get the following output:

Processing https://foo.net/pic.jpg
Downloading https://foo.net/pic.jpg ...

This is not what we expected because the process() function executes before the download() function. The correct sequence should be:

  • Download the picture and wait for the download completes.
  • Process the picture.

To resolve this issue, we can pass the process() function to the download() function and execute the process() function inside the download() function once the download completes, like this:

function download(url, callback) {
    setTimeout(() => {
        // script to download the picture here
        console.log(`Downloading ${url} ...`);
        
        // process the picture once it is completed
        callback(url);
    }, 1000);
}

function process(picture) {
    console.log(`Processing ${picture}`);
}

let url = 'https://wwww.javascripttutorial.net/pic.jpg';
download(url, process);

Output:

Downloading https://www.foo.net/pic.jpg ...
Processing https://www.foo.net/pic.jpg

Now, it works as expected.

In this example, the process() is a callback passed into an asynchronous function.

When we use a callback to continue code execution after an asynchronous operation, the callback is called an asynchronous callback.

To make the code more concise, we can define the process() function as an anonymous function:

function download(url, callback) {
    setTimeout(() => {
        // script to download the picture here
        console.log(`Downloading ${url} ...`);
        // process the picture once it is completed
        callback(url);

    }, 1000);
}

let url = 'https://www.javascripttutorial.net/pic.jpg';
download(url, function(picture) {
    console.log(`Processing ${picture}`);
}); 

Handling errors

The download() function assumes that everything works fine and does not consider any exceptions. The following code introduces two callbacks: success and failure to handle the success and failure cases respectively:

function download(url, success, failure) {
  setTimeout(() => {
    console.log(`Downloading the picture from ${url} ...`);
    !url ? failure(url) : success(url);
  }, 1000);
}

download(
  '',
  (url) => console.log(`Processing the picture ${url}`),
  (url) => console.log(`The '${url}' is not valid`)
);

Nesting callbacks and the Pyramid of Doom

How do we download three pictures and process them sequentially? A typical approach is to call the download() function inside the callback function, like this:

function download(url, callback) {
  setTimeout(() => {
    console.log(`Downloading ${url} ...`);
    callback(url);
  }, 1000);
}

const url1 = 'https://www.foo.net/pic1.jpg';
const url2 = 'https://www.foo.net/pic2.jpg';
const url3 = 'https://www.foo.net/pic3.jpg';

download(url1, function (url) {
  console.log(`Processing ${url}`);
  download(url2, function (url) {
    console.log(`Processing ${url}`);
    download(url3, function (url) {
      console.log(`Processing ${url}`);
    });
  });
});

Output:

Downloading https://www.foo.net/pic1.jpg ...
Processing https://www.foo.net/pic1.jpg
Downloading https://www.foo.net/pic2.jpg ...
Processing https://www.foo.net/pic2.jpg
Downloading https://www.foo.net/pic3.jpg ...
Processing https://www.foo.net/pic3.jpg

The script works perfectly fine.

However, this callback strategy does not scale well when the complexity grows significantly.

Nesting many asynchronous functions inside callbacks is known as the pyramid of doom or the callback hell:

asyncFunction(function(){
    asyncFunction(function(){
        asyncFunction(function(){
            asyncFunction(function(){
                asyncFunction(function(){
                    ....
                });
            });
        });
    });
});

To avoid the pyramid of doom, we use promises or async / await functions.