vendredi 19 février 2021

How can I most effectively and elegantly let the user decide which package to use for a custom framework? [closed]

I am building a framework in C++ which relies heavily on high performance mathematical operations, and want to integrate packages like Intel's MKL, CBLAS, and so forth in the framework, alongside some default operations that do not rely on any packages. I want to give the user the functionality of declaring which package to use, depending on a variable, or #define, or anything that the user can enter, and for the program to compile using those specific functions (or achieve a similar effect). Other than the identifier, rest of the program should remain the same, so it should be possible for the user to swap out various methods of packages in a single line or so. I am aware of using #ifdef's but also wondering if there are better methods for this. Also since in C++20 modules are being introduced, I am not sure how effective #ifdefs will be in the first place

Handling multiple instances/services and their disposal. Messed up design

What is the correct way to manage those instances/services? In my case, IExchangeClient should be disposed at some point and more accurately what's inside it: BinanceClient, BinanceSocketClient and Subject<IObservable<Unit>> (these 3 are third-party library instances). There are many ways where it can be disposed. I tried make it more like ASP.NET Core's way and dispose it in Main(). Maybe there is a cool NuGet package which handles that better, just like ASP.NET Core? I feel like subproject 2's design is completely messed up.

BacktestOptions, ExchangeOptions and TradeOptions are appsettings configs, in case you ask.

Backtesting subproject:

public class Backtest
{
    private readonly ITradingStrategy _tradingStrategy;
    private readonly IDataProvider _dataProvider;
    private readonly BacktestOptions _backtestOptions;
    
    public Backtest(ITradingStrategy tradingStrategy, IDataProvider dataProvider, BacktestOptions backtestOptions)
    {
        _tradingStrategy = tradingStrategy;
        _dataProvider = dataProvider;
        _backtestOptions = backtestOptions;
    }
    
    public async Task RunAsync()
    {
        ...
    }
}

public interface IDataProvider
{
    Task<List<OHLCV>> DownloadCandlesAsync(string pair, Timeframe timeframe, DateTime startDate, DateTime endDate, int startupCandleCount);
}

public class DataProvider : IDataProvider
{
    private readonly IExchangeClient _exchangeClient;

    public DataProvider(IExchangeClient exchangeClient)
    {
        _exchangeClient = exchangeClient;
    }
    
    public async Task<List<OHLCV>> DownloadCandlesAsync(string pair, Timeframe timeframe, DateTime startDate, DateTime endDate, int startupCandleCount)
    {
        ...
    }
}

public interface IExchangeClient : IDisposable
{
    ...
}

public class BinanceSpotClient : IExchangeClient
{
    private readonly IBinanceClient _client;
    private readonly IBinanceSocketClient _socketClient;
    
    public BinanceSpotClient(ExchangeOptions exchangeOptions)
    {
        _client = new BinanceClient(new BinanceClientOptions()
        {
            ApiCredentials = new ApiCredentials(exchangeOptions.ApiKey, exchangeOptions.SecretKey),
            AutoTimestamp = true,
            AutoTimestampRecalculationInterval = TimeSpan.FromMinutes(30),
            TradeRulesBehaviour = TradeRulesBehaviour.AutoComply,
#if DEBUG
            LogVerbosity = LogVerbosity.Debug
#endif
        });

        _socketClient = new BinanceSocketClient(new BinanceSocketClientOptions()
        {
            ApiCredentials = new ApiCredentials(exchangeOptions.ApiKey, exchangeOptions.SecretKey),
            AutoReconnect = true,
            ReconnectInterval = TimeSpan.FromSeconds(15),
#if DEBUG
            LogVerbosity = LogVerbosity.Debug
#endif
        });
    }
    
    private readonly Subject<IObservable<Unit>> _subject = new Subject<IObservable<Unit>>();
    
    ...
    
    private bool _disposed = false;

    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }

    protected virtual void Dispose(bool disposing)
    {
        if (_disposed)
            return;

        if (disposing)
        {
            if (_client.NotNull())
                _client.Dispose();

            if (_socketClient.NotNull())
            {
                _socketClient.UnsubscribeAll();
                _socketClient.Dispose();
            }

            _subject.OnNext(Observable.Never<Unit>());
        }

        _disposed = true;
    }
}

class Program
{
    static async Task Main(string[] args)
    {
        using IExchangeClient exchangeClient = new BinanceSpotClient(configuration.ExchangeOptions);
        ITradingStrategy tradingStrategy = StrategyUtils.GetStrategyInstance(configuration.BacktestOptions.StrategyName);
        IDataProvider dataProvider = new DataProvider(exchangeClient);
    
        var backtest = new Backtest(tradingStrategy, dataProvider, configuration.BacktestOptions);
        await backtest.RunAsync().ConfigureAwait(false);
        
        Console.ReadLine();
    }
}

Live trading subproject 2:

In this case, I have a factory method implementation, which makes the disposing harder. I'm disposing IExchangeClient in LiveTradeManager and I feel like that's terribly wrong. Maybe I should dispose IExchangeClient elsewhere and leave only the x4 web socket stream subscriptions disposed there?

public static class TradeManagerFactory
{
    public static ITradeManager Build(Manager managers, ExchangeOptions exchangeOptions, TradeOptions tradeOptions)
    {
        IExchangeClient exchangeClient = new BinanceSpotClient(exchangeOptions);
        ITradingStrategy tradingStrategy = StrategyUtils.GetStrategyInstance(tradeOptions.StrategyName);

        ITradeManager tradeManager = null;

        switch (managers)
        {
            case Manager.LiveTradeManager:
                tradeManager = new LiveTradeManager(exchangeClient, tradingStrategy, exchangeOptions, tradeOptions);
                break;
        }

        return tradeManager;
    }
}

public interface ITradeManager : IDisposable
{
    Task RunAsync();
}

public class LiveTradeManager : ITradeManager
{
    private static readonly ILog _logger = LogManager.GetLogger(MethodBase.GetCurrentMethod().DeclaringType.Name);

    private readonly IExchangeClient _exchangeClient;
    private readonly ITradingStrategy _tradingStrategy;
    private readonly ExchangeOptions _exchangeOptions;
    private readonly TradeOptions _tradeOptions;

    private readonly Wallets _wallets;
    private readonly Trades _trades;
    private readonly List<OHLCV> _candles;
    
    public LiveTradeManager(IExchangeClient exchangeClient, ITradingStrategy tradingStrategy, ExchangeOptions exchangeOptions, TradeOptions tradeOptions)
    {
        _exchangeClient = exchangeClient;
        _tradingStrategy = tradingStrategy;
        _exchangeOptions = exchangeOptions;
        _tradeOptions = tradeOptions;

        _wallets = new Wallets(exchangeClient);
        _trades = new Trades();
        _candles = new List<OHLCV>();
    }
    
    private CallResult<UpdateSubscription> _tickerSubscription;
    private CallResult<UpdateSubscription> _candlestickSubscription;
    private CallResult<UpdateSubscription> _orderSubscription;
    private IDisposable _throttlerSubscription;
    
    public async Task RunAsync()
    {
        ...
    }
    
    private bool _disposed = false;

    public void Dispose()
    {
        Dispose(true);
        GC.SuppressFinalize(this);
    }

    protected virtual void Dispose(bool disposing)
    {
        if (_disposed)
            return;

        if (disposing)
        {
            if (_exchangeClient.NotNull())
                _exchangeClient.Dispose();

            _exchangeClient.Unsubscribe(_tickerSubscription.Data);
            _exchangeClient.Unsubscribe(_candlestickSubscription.Data);
            _exchangeClient.Unsubscribe(_orderSubscription.Data);
            if (_throttlerSubscription.NotNull())
                _throttlerSubscription.Dispose();
        }

        _disposed = true;
    }
}

class Program
{
    static async Task Main(string[] args)
    {
        var tradeManager = TradeManagerFactory.Build(Manager.LiveTradeManager, configuration.ExchangeOptions, configuration.TradeOptions);
        await tradeManager.RunAsync().ConfigureAwait(false);
        
        Console.ReadLine();
    }
}

What are some architecture patterns to consider when building a REST api sync application?

TLDR;

We're syncing data by calling a third party api, mapping the data and posting to a destination api. We are integrating more third parties down the line. What are some architectural patterns to consider for this kind of software?


The Details

I'm on a team that builds a solution that sync data from one REST Api to another. We're currently only syncing from one source, to one destination. More sources will be implemented.

We started building an Onion Architecture solution, and it works just fine. However:

  • Since we only want to sync (put or post) new data, we have to call the destination api first to see what we already have. This happens in the infrastructure layer of the application.
  • Then we call the source api, and filter it so we only have new or updated data (also in infrastructure layer)
  • The data is converted from third party objects to core objects (also in infrastructure layer)
  • Then it's converted to the destination objects, and sendt to the destination api (also in infrastructure layer)

You might see where this is going..

Most data processing happens in infrastructure layer, because the core layer is independent of everything (projects, nugets, etc.). If we were to follow the Onion pattern completely, we would process in Core, but that means that we have to do more mapping before comparing data.

We all feel that a lot of what exists in the core layer, are unnecessary, since we basically just map to core objects and then map straight to the infrastructure objects.

Do you have any suggestions to other architectures we should concider?

Where should I use Factory Method?

I can't see any viable area to implement Factory Method. At its core, the design pattern offers nothing that I can't do without object initialization with new keyword. Is there any real world library that uses this pattern in a right manner?

How to give API access to entire 'office' corperations? (AWS API Gateway, Outlook VSTO)

I'm searching for a 'design' solution for the following concept:

I have a VSTO plugin for Outlook which needs to make multiple requests to 2 API's which are behind a AWS Api Gateway. These API's are processing certain items based on the corporation that is making the call.

This API needs security to hide certain information for people outside a corporation. Thats also the part where I'm stuck at right now.

The ideal situation would look like this:

  • A user can get verified based on the logged in user in Outlook.
  • A Application/API Administrator can grant & revoke API access to entire corporations.

So what prevents me from finding a solution:

  • I don't know how and if it is possible to verify a logged in Outlook user at the AWS Api Gateway auth.
  • I have looked at different solutions like JWT Tokens + Refresh Tokens, however the issue of granting access to entire corporations still exists. This also goes back to the first issue where I have to get a JWT Token for a logged in user.

So...

  1. What information can I pull out of Outlook to get a user verified at my API Auth (and maybe return JWT tokens)?
  2. Is it possible to grant a entire corporation access to your API? (So 'verified/loggged in' outlook users like bodeeh@fakecorpA.com is granted access because he is in the Office Corperation of company 'fakecorpA' but bodeeh@fakecorpB.com is not granted access because 'fakecorpB' is not granted as a Office corperation)

Is AuthenticateUserQuery a valid Query in CQRS

Recently I have been trying to implement CQRS + DDD in my new ASP.NET Core project, as CQRS and DDD are very new to me, I read a lot of articles and examples online but could not get a deeper understanding.

Generic example:

I have the following:

Structure:

Front-End <==> Controller <==> Mediatr <==> Command Handler <==> Repository

Or

Front-End <==> Controller <==> Mediatr <==> Query Handler <==> Reader

I use Mediatr library to handle all my commands and queries.

Repository returns/save Aggregate Root/Domain/WriteModel, Reader returns ReadModel.

Aggregate Root/Domain/WriteModel: class User { string UserId , string Password }

Command: CreateUserCommand which takes in new user's UserId and Password and create a new User, then save it with Repository. Publish a UserCreatedEvent.

Now I want to create a function for clients to authenticate with UserId and Password. The function should have UserId and Password as input, and I will return UserAuthResult which has a bool field, true if UserId and Password is valid

I am thinking to create a AuthenticateUserQuery but I am not sure if it is actually a valid Query, or just a Domain Service.

The code flow of AuthenticateUserQuery would probably be:

  1. Controller send AuthenticateUserQuery to Mediatr
  2. Mediatr execute AuthenticateUserQueryHandler.Handle()
  3. AuthenticateUserQueryHandler get UserAuthDetailReadModel from UserAuthDetailReader
  4. Do authenticate with query's UserId + Password and UserAuthDetailReadModel
  5. Return UserAuthResult

Questions Time!

  1. From the description above, AuthenticateUserQuery only performs query/read, is it a valid Query or Domain Service
  2. If Question #1 is Query, is UserAuthResult a ReadModel? (Since it is returned from a Query)
  3. If Question #2 is a YES, is UserAuthDetailReadModel also a ReadModel? (Since it is NOT returned from Query, but return from Reader)
  4. If Question #1 is Query, does it mean as long as the Query only performs query/read with ReadModel, the name of Query does not limit to CRUD style of prefix (e.g. GetUserAuthDetailQuery)?
  5. If Question #1 is Domain Service, would you please give me some idea on how to implement it? (e.g. Class + Method signature)
  6. If Question #1 is neither, would you please give me some insight?

Appreciate any related suggestions and ideas. Thank you!

jeudi 18 février 2021

How buid a django architecture for multiple user interfaces working with the same data

I have a model like Company. Two user interfaces: an administrator's cabinet(Cabinet) (not native Django) and an interface for ordinary users (Main).

Let's use the example of a list of companies. On the Cabinet, I want to show the companies that he can edit, on the Main - all public companies. I doubt for how to properly organize this moment on the backend.

What I think? Make the base class CompanyListService in the service layer, write general logic in it, and separately each interface has its own class: CabinetCompanyListService(CompanyListService), MainCompanyListService(CompanyListService)

Pass the model manager to it during initialization(Company is Django Model):

class CompanyListService:
    def __init__(self, manager=Company.main_manager):
        self.manager = manager

then I can get here all the data options needed for the front

class MainCompanyListService(CompanyListService):

    def get_companies(self):
        return self.manager.get_publish_companies()


class CabinetCompanyListService(CompanyListService):

    def get_companies(self):
        return self.manager.get_available_companies_where_user_is_staff()


class MainManager(TreeManager):
    def get_available_companies(self):
        """Includes blocked, rejected etc...
        Companies that actually exist and are available at least to the creator, admin"""
        return self.get_queryset().exclude(status='deleted')

    def get_publish_companies(self):
        return self.get_queryset(status='active')

    def get_available_companies_where_user_is_staff(self, user):
        """Companies where user has CompanyManager role"""
        self.get_available_companies().filter(company_company_managers__user=user)

Not sure if this is good architecture. In general, I don't have enough experience to understand how good or bad it is to do this. How do you do this kind of thing? It is with Django. Thanks!