mercredi 18 septembre 2019

Identify patterns within list of words with pattern threshold

Working on a pattern recognition function in Python that suppose to return an array of patterns with a counter

Let's imagine a list of strings:

m = ['ABA','ABB', 'ABC','BCA','BCB','BCC','ABBC', 'ABBA', 'ABBC']

at the high-level, what I would like to get back is:

Pattern | Count
----------------
   AB   |   6
  ABB   |   3
   BC   |   2
----------------

The problem: all I know that patterns begin with 2 characters and are leading characters for each string value (i.e. XXZZZ, XXXZZZ (where XX is a pattern that I'm looking for)). I would like to be able to parametrize minimal length of a pattern as a function's input to optimize the run time.

PS. each item in the list is a single word already.

my problem is that I need to iterate for each letter starting from the threshold, and I'm getting stuck there. I'd prefer to use startswith('AB')

Node.js send message from worker on exit event

I have a Node application creating a number of worker processes via fork() to perform some tasks on their own. As a byproduct, the workers produce a snapshot of actions taken in the form of object. Each worker has an event listener attached to the 'exit' event, at which time they send their snapshot back to the parent process via process.send().

Here's an example of the set up:

// parent.js
const exec = require('child_process');

const worker = exec.fork('worker.js', [], {
  stdio: ['pipe', 'pipe', null, 'ipc']
});

worker.on('message', (snapshot) => {
  // Handle the snapshot sent from worker
});

// worker.js
process.on('exit', () => {
  process.send({ /* snapshot data */ })
});
/* Arbitrary task work */

Is this an acceptable pattern for parent/worker IPC? Specifically, will this reliably about the reliability of receiving messages in the parent process given the nature of the exit event and process.send()?

Is the beforeExit event better suited to this pattern?

Mapping abstraction?

There will be a huge object mapping (transforming) with a lot of calculations and logic for some fields.

I would like to refactor this to classes or some kind of separation so it will be easily testable (jest) for each field. How would you abstract these to a few classes or which design pattern that is best suited for this?

For example:

function mapData(data) {
  const orderItems = data.items.map(item => {
    return {
      Name: item.title,
      Qty: parseInt(item.qty),
      Price: parseFloat(item.price),
      PriceExcVat: getItemPriceExcVat(item),
      TotalIncVat: getItemTotalIncVat(item),
      TotalExcVat: getItemTotalExcVat(item),
      // and many more fields
    }
  });

  return {
     Id:  data.data,
     Total: data_grand_total,
     TotalIncVat: getOrderIncVat(data),
     Items: orderItems,
     DeliveryAddress: {
       Name: data.address.name,
       Address1: data.address.line_1,
       Address2: data.address.line_2,
       Address3: data.address.line_3,
     },
      // and many more fields
  }
}

function getOrderIncVat(data) { return /*do some math calculation logic here*/ }
function getItemPriceExcVat(data) {  return /*do some math calculation logic here*/ }
function getItemTotalIncVat(data) {  return /*do some math calculation logic here*/ }
function getItemTotalExcVat(data) {  return /*do some math calculation logic here*/ }

How can I adapt a regex pattern for VBA

I am trying to remove a specific string of text using regex and have the following which works on regex101.com however i understand that VBA has some nuances when using regex and the pattern i have doesn't seem to work at all.

the pattern which i validated is in the code below, If possible i would also like to combine it with the below (designed to remove html tags) "\<.*?>" this does work but i currently run a separate regex.replace function to remove this.

Dim regEx As Object , str As String
Set regEx = CreateObject("VBScript.RegExp")

With regEx
  '.Pattern = "<a\b([^>""']|""[^""]*""|'[^']*')+class=""changed-by"">.*?<\/a>"
'.pattern = "\<.*?\>" ' this is the second pattern i use to remove html tages which works
  .Global = True 'If False, would replace only first
  .IgnoreCase = False
  .MultiLine = False
End With

str = some text here <a href="/instrument/2014/36.pdf" target="_blank" title="2014/36 - 01/07/2014" class="changed-by">1</a> some text here
Debug.Print regEx.Replace(str, "")

How to resolve dependency graph loop

I have Foo and Bar classes. I want to create FooForBar class wich provides acces to Foo instances by specific way (want to limit acces to specific fields of Foo instances). In the same time it needs to store Bar instances in Foo instances. And there is a loop in dependencies graph wich is not good.

uml

How can I resolve it or make this architecture another way?

Best approach/language for batch applications

I'm thinking to develop a new application. But I need help in order to understand which is the best approach/language to use with this architecture:

  • Few minor console applications that periodically retrieve data from api services and store them into database.
  • Main console application that manage all the process. I mean, invoke previous minor console applications, process data (saving on database). During processing, the application selects few records (~10-50). For each of them I need to execute some actions. Those actions should be repeated frequently (for example, every 10 seconds for one or two hours). For those actions some apis should be invoked, and result saved on db.
  • Supervisor monitor based on web application. Show in real time data collected on db. (Each console applications save data on db, so from monitor I wish to get applications status).

Thank you

The design pattern behind the SNAFU library

I've been playing around with the interesting SNAFU library.

A slightly modified and stand-alone example from the SNAFU page is as below:

use snafu::{ResultExt, Snafu};
use std::{fs, io, path::PathBuf};

#[derive(Debug, Snafu)]
enum Error {
    #[snafu(display("Unable to read configuration from {}: {}", path.display(), source))]
    ReadConfiguration { source: io::Error, path: PathBuf },
}

type Result<T, E = Error> = std::result::Result<T, E>;

fn process_data() -> Result<()> {
    let path = "config.toml";
    let read_result: std::result::Result<String, io::Error> = fs::read_to_string(path);    
    let _configuration = read_result.context(ReadConfiguration { path })?;
    Ok(())
}

fn main() {
    let foo = process_data();

    match foo {
        Err(e) => println!("Hello {}", e),
        _ => println!("success")
    }
}

The change I've made is to make the type on the Result from fs::read_to_string(path) explicit in process_data().

Given this, I can't understand how read_result has the context method available to it, as the std::result::Result docs don't make any reference to context (and the compiler similarly complains if you strip out the SNAFU stuff and try to access context).

There is a pattern being used here that is not obvious to me. My naive understanding is that external types cannot be extended because of the orphan rules, but something is happening here that looks much like such an extension.

I'm also confused by the type Result... line. I'm aware of type aliasing, but not using the syntax in which the left hand side has a generic assigned. Clearly, this is an important part of the design pattern.

My request is for clarification as to what pattern is being used here and how it works. It seems to get at some pretty interesting aspects of Rust. Further reading would be valued!