MariaDB Insert Large Dataset (Java)

EntityManager entityManagerA = entityManagerFactoryA.createEntityManager();
QPayment payment = QPayment.payment;
JPAQueryFactory qf = new JPAQueryFactory(entityManagerA);
JPAQuery<PaymentResponse> query = qf.from(payment)
        .groupBy(payment.refnum)
        .select(
                Projections.bean(
                        PaymentResponse.class,
                        payment.refnum.as(PaymentResponse.PaymentResponse_.REFNUM),
                        payment.idtrx.as(PaymentResponse.PaymentResponse_.IDTRX),
                        payment.jenisTransaksi.as(PaymentResponse.PaymentResponse_.PRODUK),
                        payment.tglbayar.as(PaymentResponse.PaymentResponse_.TANGGAL_BAYAR),
                        payment.rptag.castToNum(Long.class).add(payment.rpadmin.castToNum(Long.class)).sum().stringValue().as(PaymentResponse.PaymentResponse_.AMOUNT)
                )
        ).where(payment.tglbayar.between(from, to).and(payment.kodebank.in(banks)))
        .orderBy(payment.tglbayar.desc());

String start = DateTimes.format(from);
String end = DateTimes.format(to);
String file = System.getProperty("user.dir") + "/" + start + "-" + end + "-" + System.currentTimeMillis() + ".csv";
LocalDateTime now = DateTimes.now();
log.info("Started...");
log.info("Writing to file: {}", file);
final AtomicLong total = new AtomicLong();
try (RandomAccessFile writer = new RandomAccessFile(file, "rw");
     FileChannel channel = writer.getChannel()) {
    final ByteBuffer buff = ByteBuffer.allocate(65535);
    query.stream().forEach(paymentResponse -> {
        buff.clear();
        String line = String.format("%s;%s;%s;%s;%d;%s\n", paymentResponse.getAmount(), DateTimes.format(paymentResponse.getTanggalBayar()), paymentResponse.getIdtrx(), paymentResponse.getProduk(), pullPlnId, paymentResponse.getRefnum());
        buff.put(line.getBytes(StandardCharsets.UTF_8));
        buff.flip();
        try {
            channel.write(buff);
            total.incrementAndGet();
        } catch (IOException e) {
            // do something
        }
    });
} catch (FileNotFoundException e) {
    // do something
} catch (IOException e) {
    // do something
} finally {
    entityManagerA.close();
}

EntityManager entityManagerB = entityManagerFactoryB.createEntityManager();

Session session = entityManagerB.unwrap(Session.class);
session.doWork(connection -> {
    String baQuery = "LOAD DATA LOCAL INFILE ? INTO TABLE my_dst_table FIELDS TERMINATED BY ';' LINES TERMINATED BY '\n'";
    PreparedStatement preparedStatement = connection.prepareStatement(baQuery);
    preparedStatement.setString(1, file);
    boolean execute = preparedStatement.execute();
    System.out.println("EXECUTED: " + execute);
});
long between = ChronoUnit.MINUTES.between(now, DateTimes.now());
log.info("Done: takes {} minutes.", between);
entityManagerB.close();

*) Inserting more then 400k data takes less the a minutes.

Refs:

https://mariadb.com/kb/en/how-to-quickly-insert-data-into-mariadb/

https://mariadb.com/kb/en/load-data-infile/

https://mariadb.com/kb/en/unsafe-statements-for-statement-based-replication/

Prime Factor

Simple prime factorization

import math

def prime_factor(num):
    factors = []
    while num % 2 == 0:
        factors.append(2)
        num = num / 2
    for i in range(3, int(math.sqrt(num)) + 1, 2): # 3, 5, 7, ...
        while num % i == 0:
            factors.append(i)
            num = num / i
    if num > 2:
        factors.append(num)
    return factors


num = 1000003 + 1
factors = prime_factor(num)

if len(factors) == 1:
    print(str(num) + " is prime")
else:
    print(str(num) + " = multiply(" + str(factors) + ")")

Mikrotik Wireless Client Mode

Reset

/system reset-configuration

Login

Scan

/interface wireless scan wlan1

Configure

/interface wireless security-profiles
add authentication-types=wpa-psk,wpa2-psk \
    management-protection=allowed \
    mode=dynamic-keys \
    unicast-ciphers=tkip,aes-ccm \
    group-ciphers=tkip,aes-ccm \
    name=Bless2Profile \
    supplicant-identity=Bless2 \
    wpa-pre-shared-key=mysecretpassword \
    wpa2-pre-shared-key=mysecretpassword

/interface wireless
set [ find default-name=wlan1 ] \
    disabled=no \
    mode=station \
    security-profile=Bless2Profile \
    ssid=Bless2

/ip dhcp-client
add comment=defconf \
    add-default-route=yes \
    disabled=no \
    interface=wlan1

Remove wlan1 from bridge interface

/interface bridge port print
/interface bridge port remove numbers=1

Add ether1-4 to bridge interface

/interface bridge port add bridge=bridge interface=ether2
/interface bridge port add bridge=bridge interface=ether3
/interface bridge port add bridge=bridge interface=ether4

NAT

/ip firewall nat add chain=srcnat action=masquerade out-interface=wlan1