Async Task

The Async_Task table is useful for keep information on asynchronous tasks. Asynchronous tasks are useful when the processing time of a business operation will take longer than what Azure managed services (such as Azure App Service or Azure Front Door) allow. For example, Azure App Service has a maximum timeout of 230 seconds (src) so if you have a back-end method that takes almost that long or more, then you’ll need to make the call to the method asynchronous, meaning the front end doesn’t wait for the back end to finish executing the method. This style of operation is called asynchronous messaging pattern.

A unique task is a long-lived task where the task can only be executed once at any time per customer. For example, the following record must be inserted first; change 'Foo Task' in the IF statement and in the INSERT statement:

IF (NOT EXISTS (SELECT 1 FROM FooNovus.dbo.Async_Task WHERE name = 'Foo Task'))
BEGIN
  INSERT INTO FooNovus.dbo.Async_Task (name,status,created_on,created_by,version) VALUES (
    'Foo Task', -- name,
    'Idle', -- status,
    SYSUTCDATETIME(), -- created_on,
    'admin', -- created_by,
    0 -- version
  );
END

The task’s status starts at Idle. When the user or system triggers the task, the task’s status is changed to Execute, and a message will be inserted into an Azure Queue (in an Azure Storage Account resource). An Azure function will then dequeue the message, change the task’s status to Executing, execute the business logic, and then change the tasks’ status back to 'Idle'.

(Unique task) Add a new back-end method to insert message into Azure queue. First, add a method to check if the task is completed (i.e. status is Idle):

private async Task<bool> IsFooTaskCompleted()
{
  return await dbContext.AsyncTask
    .AsNoTracking()
    .AnyAsync(p => p.Name == AsyncTask.TaskFoo
      && p.Status == AsyncTask.StatusIdle
    );
}

Add the new back-end method. The example below shows an example where the user may select a set of foo IDs to pass to the business operation. public async Task<BusinessResult> QueueFoo(

 List<int> fooIds
 )

{

 var result = new BusinessResult();
 result.ResultStatus = ResultStatus.Success;

 if (!await IsFooTaskCompleted())
 {
   result.ResultCode = "1";
   return result;
 }

 var userId = serviceHelper2.GetUserId();

 var queueMessage = new CustomQueueMessage();
 queueMessage.Tenant = (await tenantProvider.GetTenant()).GetAsDTO();
 queueMessage.UserId = userId;

 var queueMessageJson = JsonConvert.SerializeObject(queueMessage);

 var taskDataObject = new FooTaskData();
 taskDataObject.FooIds = fooIds;
 var taskData = JsonConvert.SerializeObject(taskDataObject);

 /* execute job and queue up foo message */

 BusinessResult insertMessageResult = null;

 using (var transaction = await dbContext.Database.BeginTransactionAsync())
 {
   try
   {
     var dbConn = dbContext.Database.GetDbConnection();
     if (dbConn.State != ConnectionState.Open)
     {
       await dbConn.OpenAsync();
     }

     var updated = await dbConn.QueryFirstOrDefaultAsync<bool>(
       "usp_UpdateAsyncTaskStatusWithDetails",
       new
       {
         asyncTaskName = AsyncTask.TaskFoo,
         currentStatus = AsyncTask.StatusIdle,
         userId,
         taskData,
       },
       commandType: CommandType.StoredProcedure,
       transaction: transaction.GetDbTransaction()
       );

     transaction.Commit();

     if (updated)
     {
       insertMessageResult = await queueService.InsertMessage("novus-foo-queue", queueMessageJson);
     }
   }
   catch (Exception ex)
   {
     transaction.Rollback();
     throw ex;
   }
 }

 /*
  * check if foo message was able to be queued and if not,
  * then clear out Async_Task_Details by updating the Async_Task
  * to Idle.
  */

 if (insertMessageResult != null && insertMessageResult.ResultStatus != ResultStatus.Success)
 {
   using (var transaction = await dbContext.Database.BeginTransactionAsync())
   {
     try
     {
       var dbConn = dbContext.Database.GetDbConnection();
       if (dbConn.State != ConnectionState.Open)
       {
         await dbConn.OpenAsync();
       }

       await dbConn.QueryAsync<string>(
         "usp_AsyncTaskMarkAsIdle",
         new
         {
           asyncTaskName = AsyncTask.TaskFoo,
           userId
         },
         commandType: CommandType.StoredProcedure,
         transaction: transaction.GetDbTransaction()
         );

       transaction.Commit();

       result.CopyTheResultOf(insertMessageResult);
       return result;
     }
     catch (Exception ex)
     {
       transaction.Rollback();
       throw ex;
     }
   }
 }

 return result;

}

Depending on the environment (dev/QA, UAT, or prod), the name of the queue is novus-foo-queue + environment e.g. novus-foo-queue-dev. For the front end to check if the task is completed, add the following back-end service method and API controller method:

public async Task<object> CheckFooTask()
{
  return new
  {
    completed = await IsFooTaskCompleted(),
  };
}
[HttpGet("_check-foo-task")]
public async Task<ActionResult<bool>> CheckFooTask()
{
  object taskStatus = await fooService.CheckFooTask();
  return Ok(taskStatus);
}

If you’re refactoring a business operation that was synchronous before, then add a new controller method that inserts into the queue besides the old method e.g.

[Route("process")]
[HttpPost]
public async Task<BusinessResult> Foo([FromBody] List<int> fooIds)
{
  return await this.fooService.Foo(fooIds);
}

[Route("queue-process")]
[HttpPost]
public async Task<BusinessResult> QueueFoo([FromBody] List<int> fooIds)
{
  Return await this.fooService.QueueFoo(fooIds);
}

(Unique task) Front-end changes. In general, the front end would start a background interval that periodically checks if the task is currently being executed and if it is, disable everything on the form and show a popup message. If the back-end business operation is executed, then show the popup message immediately.

Things may get a little trickier if you need to refactor an existing business operation that is timing out by applying the asynchronous messaging pattern to it, because you may have to refactor other logic that is particular to the form e.g. refactor the callback since the business operation is not synchronous. Another issue to refactoring an existing business operation is that there might be infrastructure changes on the Azure side or the customer may want additional changes before going live with the Azure function way, so a feature toggle may be necessary by inserting the below configuration:

INSERT INTO FooNovus.dbo.configuration (control_id,description,control_value,version) VALUES (
  'FooProcess Use Azure Function', -- control_id,
  'If True, then the Foo Process page will use an Azure function to process Foo.', -- description,
  'False', -- control_value,
  0 -- version
);

By default, if the configuration doesn’t exist, or if the control_value is False, then the old non-asynchronous way will be used. Add the following to 'src/i18n-messages.ts:'

fooCompleted: 'The foo process is completed.',
fooTaskInProgress: `The foo process is running. Please wait for it to finish. This dialog will close automatically when it's done.`,

In the Vue page’s template:

<b-modal
  v-model="isFooTaskStatusModalVisible"
  size="md"
  centered
  no-enforce-focus
  no-close-on-backdrop
  no-fade
  auto-focus-button="ok"
  ok-only
>
  {{ $t('message.fooTaskInProgress') }}
</b-modal>

To every UI element that can be disabled on the form, add

 :disabled=

"

!isFooTaskCompleted"

so that when the task is executing, all UI elements on the form are disabled until the task is completed.

In the Vue page, add the following:

import qs from 'qs';
import Configuration from '@/models/Configuration';
import { BusinessResult } from '@/common-axios';

type TaskStatus = {
  completed: boolean
}

@Component({
  // ...
  private useAzureFunction = false;
  private useAzureFunctionPromise: Promise<boolean> | null = null;

  public activated() {
    this.shellModule.setPendingMenuRouteComponent(this);

    this.checkIfAzureFunctionIsUsed()
    .then(() => {
      if (this.useAzureFunction) {
        this.checkFooTaskCompleted(true);
      }
    });
  }

  private checkIfAzureFunctionIsUsed() {
    if (this.useAzureFunctionPromise) {
      // this check ensures `useAzureFunction` is only checked once
      return this.useAzureFunctionPromise;
    }

    const queryString = qs.stringify({
      controlId: 'FooProcess Use Azure Function',
    });

    this.useAzureFunctionPromise = this.getData<Configuration | null>(`Configuration/GetConfigurationByControlId?${queryString}`)
    .then((configuration) => {
      configuration = configuration ? Configuration.fromJson(configuration) : null;

      this.useAzureFunction = configuration != null
        && configuration.ControlValue != null
        && configuration.ControlValue.toUpperCase() === 'TRUE';

      return this.useAzureFunction;
    });

    return this.useAzureFunctionPromise;
  }

  public deactivated() {
    this.setupFooTaskCompleteChecker(false);
  }

  public softDestroy() {
    // ...

    this.useAzureFunction = false;
    this.useAzureFunctionPromise = null;

    this.fooTaskCompleteCheckerIntervalId = null;
    this.isFooTaskStatusModalVisible = false;
    this.isFooTaskCompleted = true;
    this.checkFooTaskPromise = Promise.resolve({ completed: true });
  }

  public fooTaskCompleteCheckerIntervalId: number | null = null;
  public isFooTaskStatusModalVisible = false;
  public isFooTaskCompleted = true;
  public checkFooTaskPromise: Promise<TaskStatus> = Promise.resolve({ completed: true });
  public setupFooTaskCompleteChecker(setIntervalToCheck = true) {
    if (setIntervalToCheck) {
      if (!this.fooTaskCompleteCheckerIntervalId) {
        this.fooTaskCompleteCheckerIntervalId = setInterval(() => {
          this.checkFooTaskCompleted();
        }, 30000);
      }
    } else if (this.fooTaskCompleteCheckerIntervalId) {
      clearInterval(this.fooTaskCompleteCheckerIntervalId);
      this.fooTaskCompleteCheckerIntervalId = null;
    }
  }

  public checkFooTaskCompleted(forceSetupInterval = false) {
    this.checkFooTaskPromise = this.getData<TaskStatus>('foo/_check-foo-task')
    .then((taskStatus) => {
      const wasInProgress = !this.isFooTaskCompleted && taskStatus.completed;

      this.isFooTaskCompleted = taskStatus.completed;

      if (taskStatus.completed) {
        /*
         * bug-719 Do not clear interval even though the task is completed because someone
         * else may run the Foo Process while you're still on the page.
         */
        // if (this.fooTaskCompleteCheckerIntervalId) {
        //   clearInterval(this.fooTaskCompleteCheckerIntervalId);
        // }
        if (forceSetupInterval) {
          this.setupFooTaskCompleteChecker();
        }

        this.isFooTaskStatusModalVisible = false;
      
        if (wasInProgress) {
          this.showGenericModal(this.$t('message.fooCompleted').toString());
          this.getTableData();
        }
        return taskStatus;
      }

      this.setupFooTaskCompleteChecker();
      this.isFooTaskStatusModalVisible = true;

      return taskStatus;
    });
  }

Call the back-end business operation depending on whether to use Azure function based on the configuration or use the old way:

if (this.useAzureFunction) {
  this.postData<BusinessResult>("foo/queue-process", postData)
  .then((result) => {
    this.shellModule.setShowLoading(false);
    result = BusinessResult.fromJson(result);

    if (result.ResultStatus !== BusinessResult.StatusSuccess) {
      this.isProcessButtonEnabled = true;
      this.showGenericModal(result.Message || '', 'Error');
      return;
    }

    this.isFooTaskCompleted = true;
    this.isFooTaskStatusModalVisible = true;

    this.checkFooTaskCompleted(true);

    this.getTableData();
    this.isProcessButtonEnabled = true;
  })
  .catch((err: any) => {
    this.shellModule.setShowLoading(false);
    this.isProcessButtonEnabled = true;
    console.log(err);
  });
} else {
  /* old way i.e. not using Azure function */

  this.postData<BusinessResult>("foo/process", postData)
  .then((result) => {
    this.shellModule.setShowLoading(false);
    result = BusinessResult.fromJson(result);

    if (result.ResultStatus !== BusinessResult.StatusSuccess) {
      this.isProcessButtonEnabled = true;
      this.showGenericModal(result.Message || '', 'Error');
      return;
    }

    this.getTableData();
    this.isProcessButtonEnabled = true;
  })
  .catch((err: any) => {
    this.shellModule.setShowLoading(false);
    this.isProcessButtonEnabled = true;
    console.log(err);
  });
}

(Unique task) Add Azure function. Right-click 'Novus.AzureFunction > Add > New Azure Function', rename the function to 'FooQueueConsumer.cs', click Add, on the left select Queue trigger, set connection string setting name to AzureStorageConnectionString, set queue name to 'novus-foo-queue-dev', and click Add. In the 'local.settings.json', add "FooQueueName": "novus-foo-queue-dev".

Refactor FooQueueConsumer.cs to the below:
public class FooQueueConsumer
{
  private readonly ITenantProvider tenantProvider;
  private readonly IUserProvider userProvider;
  private readonly IFooService fooService;
  private readonly NOVUSContext dbContext;

  public FooQueueConsumer(
    ITenantProvider tenantProvider,
    IUserProvider userProvider,
    IFooService fooService,
    NOVUSContext dbContext
    )
  {
    this.tenantProvider = tenantProvider;
    this.userProvider = userProvider;
    this.invoiceService = invoiceService;
    this.dbContext = dbContext;
  }

  [FunctionName("FooQueueConsumer")]
  public async Task Run(
    [QueueTrigger("%FooQueueName%", Connection = "AzureStorageConnectionString")]
    string myQueueItem,
    ILogger log
    )
  {
    try
    {
      var queueMessage = JsonConvert.DeserializeObject<FooQueueMessage>(myQueueItem);
      queueMessage.Tenant.DecryptDatabaseLogins();

      ((QueueMessageTenantProvider)tenantProvider).Tenant = queueMessage.Tenant;
      ((QueueMessageUserProvider)userProvider).UserId = queueMessage.UserId;

      /* mark AsyncTask as executing */

      var willBeExecutingTask = false;

      using (var transaction = await dbContext.Database.BeginTransactionAsync())
      {
        var dbConn = dbContext.Database.GetDbConnection();
        if (dbConn.State != ConnectionState.Open)
        {
          await dbConn.OpenAsync();
        }

        willBeExecutingTask = await dbConn.QueryFirstOrDefaultAsync<bool>(
          "usp_UpdateAsyncTaskStatusWithDetails",
          new
          {
            asyncTaskName = AsyncTask.TaskFoo,
            currentStatus = AsyncTask.StatusExecute,
            userId = userProvider.GetUserId(),
          },
          commandType: CommandType.StoredProcedure,
          transaction: transaction.GetDbTransaction()
          );

        await transaction.CommitAsync();
      }

      if (willBeExecutingTask)
      {
        /* extract task data and call business operation */

        var asyncTask = await dbContext.AsyncTask
          .AsNoTracking()
          .Where(p => p.Name == AsyncTask.TaskFoo)
          .FirstOrDefaultAsync();
        var taskDataObject
          = JsonConvert.DeserializeObject<FooTaskData>(asyncTask.TaskData);

        await fooService.Foo(taskDataObject.FooIds);
      }
    }
    finally
    {
      dbContext.ClearChanges(); // to prevent saving left over changes in ProcessFoo()

      /* mark AsyncTask as completed */

      using (var transaction = await dbContext.Database.BeginTransactionAsync())
      {
        var dbConn = dbContext.Database.GetDbConnection();
        if (dbConn.State != ConnectionState.Open)
        {
          await dbConn.OpenAsync();
        }

        await dbConn.QueryAsync<string>(
          "usp_AsyncTaskMarkAsIdle",
          new
          {
            asyncTaskName = AsyncTask.TaskFoo,
            userId = userProvider.GetUserId() ?? "system"
          },
          commandType: CommandType.StoredProcedure,
          transaction: transaction.GetDbTransaction()
          );

        await transaction.CommitAsync();
      }
    }
    
    log.LogInformation($"InvoiceQueueConsumer function processed: {myQueueItem}");
  }
}

(LEGACY) There’s an Async_Task_Detail table which provides additional detail information for an Async_Task record which serves as the header for the details. However, it is discouraged from being used on new development. Instead, serialize C# object into JSON string and store JSON string into 'Async_Task.task_data' column which has an unlimited size.