Skip to main content
Version: Previous

Day three

Day two recap
Here are the main takeaways from Day two.
  • We provided an introduction to UI, describing Web Components and Micro front-ends.
  • The Genesis UI packages​ were presented
  • We created a user interface using the Micro front-end EntityManagement.
  • We extended the application by adding new tables and CRUD events.
  • This day covers:

    Views

    When you set up a data model, it implies relationships between tables. For example, a TRADE has a COUNTERPARTY_ID and an INSTRUMENT_ID. That means it has a relationship with the COUNTERPARTY and INSTRUMENTS tables.

    Views enable you to join related tables to create a single holistic view.

    In short, Views are the genesis equivalent of SQL select queries. Unlike tables, views do not have any data of their own, they are read-only, but present a view based on one or more tables.

    A view always starts with a single table, the root table. Other tables can be joined onto the root table to present composite data.

    Views are very powerful and in this training we're going to cover just the basics. When you have a chance, try to look at the documentation.

    Entities

    During code generation, view and index entities will be generated from the definitions in your application's view-dictionary.kts file. The name of each entity will be the same as the definition, but it is converted from snake case to camel case; for example, VIEW_NAME becomes ViewName.

    The generated entities are kotlin data classes and can be built using the primary constructor (so you can also import Views in your Java/Kotlin code as well). Just before the object is built, it is validated to make sure all required fields have been set.

    Usage

    We are going to use the file alpha-view-dictionary.kts in the folder server/alpha-app/src/main/genesis/cfg/.

    The example below creates a view called TRADE_VIEW, which joins the TRADE table to the INSTRUMENT table. Edit the alpha–view-dictionary.kts file and add a view on the TRADE table:

    views {

    view("TRADE_VIEW", TRADE) {

    joins {
    joining(INSTRUMENT) {
    on(TRADE.INSTRUMENT_ID to INSTRUMENT { INSTRUMENT_ID })
    }
    }

    fields {
    TRADE.allFields()

    INSTRUMENT.INSTRUMENT_NAME
    INSTRUMENT.MARKET_ID withPrefix INSTRUMENT
    INSTRUMENT.CURRENCY_ID withAlias "CURRENCY"
    }
    }
    }
    withPrefix and withAlias

    withPrefix adds a prefix to the standard field name. For example, INSTRUMENT.MARKET_ID withPrefix SYMBOL becomes SYMBOL_MARKET_ID.

    withAlias gives the field an alternative name for the view.

    More info here.

    Now go to the Data Server definition (open alpha-dataserver.kts). Replace the ALL_TRADES query in the Data Server with the new TRADE_VIEW.

    dataServer {
    query("ALL_TRADES", TRADE_VIEW)
    ...
    }
    tip

    In the example above, you are exposing a view through a Data Server query. It's also possible to inject a view into a Request Server or even your Event Handler code. This makes it easier to access complex data from multiple tables in your Kotlin or Java code. Look at package global.genesis.gen.view.repository.

    Run build and deploy, and test the view with Postman or Console.

    Exercise 3.1: using views

    ESTIMATED TIME

    30 mins

    Extend the TRADE_VIEW to connect TRADE to COUNTERPARTY:

    1. Add the respective join (as we did with INSTRUMENT).​
    2. Add the field COUNTERPARTY.COUNTERPARTY_NAME.
    3. Test it.

    Extending our application further

    Moving on, for our app to be able to keep positions based on the trades, we now need to extend our data model.

    Adding new fields and tables

    Let's add new fields to the Trade table.

    alpha-tables-dictionary.kts
    tables {

    table (name = "TRADE", id = 2000) {
    ...

    field("TRADE_DATE", type = DATE)
    field("ENTERED_BY", type = STRING)
    field(name = "TRADE_STATUS", type = ENUM("NEW", "ALLOCATED", "CANCELLED", default = "NEW"))

    }
    ...
    }

    And new fields to create the POSITION and INSTRUMENT_PRICE tables:

    alpha-tables-dictionary.kts
        table(name = "POSITION", id = 2003) {
    field("POSITION_ID",STRING).sequence("PS").primaryKey() //autogenerated sequence
    field("INSTRUMENT_ID",STRING).notNull().uniqueIndex()
    field("QUANTITY",INT)
    field("NOTIONAL",DOUBLE)
    field("VALUE",DOUBLE)
    field("PNL",DOUBLE)
    }
    alpha-tables-dictionary.kts
        table(name = "INSTRUMENT_PRICE", id = 2004) {
    field("INSTRUMENT_ID",STRING).primaryKey()
    field("LAST_PRICE",DOUBLE)
    }

    Since we have modified the tables definition (added two new tables), we need to build and run the remap again.

    Updating the schemas.ts

    After creating these new fields, go back to the schemas.ts and add this code blocks to it. so we can interact with them.

    schemas.ts
    const conditionalSchemaEntry = (predicate: boolean, entry) => {
    return predicate ? [entry] : [];
    };

    export const tradeFormSchema = (editing?: boolean) => ({
    ...
    {
    "type": "Control",
    "label": "Counterparty",
    "scope": "#/properties/COUNTERPARTY_ID",
    "options": {
    allOptionsResourceName: "ALL_COUNTERPARTIES",
    valueField: "COUNTERPARTY_ID",
    labelField: "COUNTERPARTY_NAME",
    data: null,
    },
    },
    ...
    {
    "type": "Control",
    "label": "Trade Date",
    "scope": "#/properties/TRADE_DATE"
    },
    {
    "type": "Control",
    "label": "Status",
    "scope": "#/properties/TRADE_STATUS"
    },
    {
    "type": "Control",
    "label": "Entered By",
    "scope": "#/properties/ENTERED_BY"
    }

    ],
    });

    export const tradeFormCreateSchema = tradeFormSchema(false);
    export const tradeFormUpdateSchema = tradeFormSchema(true);
    tip

    Note that Genesis provides several autogenerated primary keys: sequence, uuid, autoIncrement.

    Automated testing

    So far we have been testing our work manually, using Genesis Console or some HTTP client. Now the time has come to start writing some automated tests for our application. We are going to test our TradeView and the Trade insert method we created.

    Configuration

    To test our classes we need to mock the database, as there are integrations and configurations managed by Genesis behind the scenes. To avoid any additional installation locally, we shall use the H2 in-memory database.

    You already have the configuration needed if you cloned the Developer Training starting repo from here. Otherwise, change the server/alpha-app/build.gradle.kts configuration for the tests tasks to match the code below.

    server/alpha-app/build.gradle.kts
    dependencies {
    compileOnly(genesis("script-dependencies"))
    genesisGeneratedCode(withTestDependency = true)
    testImplementation("global.genesis:genesis-testsupport")
    testImplementation("global.genesis:genesis-dataserver2")
    testImplementation("global.genesis:genesis-pal-dataserver")
    }

    Adding Testing: AlphaTradeViewTest

    Let's create an automated test that inserts and retrieves some data using the platform's automated test support components. We are extending the class AbstractDatabaseTest to allow a proper integration testing, as well as using the TradeView we created to assert results.

    In summary, the new test will:

    So, first, let's do the following:

    1. Create a new folder called test/kotlin under the server/alpha-app/src/
    2. Add a new package called global.genesis to the test/kotlin/ directory.
    3. Add a new test class called AlphaTradeViewTest.kt.
    4. Add a new package called resources.data.
    5. Add TEST_DATA_VIEW.csv to that package (/server/alpha-app/src/test/kotlin/resources/data).
    TEST_DATA_VIEW.csv
    #INSTRUMENT
    INSTRUMENT_ID,INSTRUMENT_NAME
    1,FOO.L
    2,BAR.L
    #COUNTERPARTY
    COUNTERPARTY_ID,COUNTERPARTY_LEI,COUNTERPARTY_NAME,
    1,335800A8HK6JBITVPA30,Test Ltd,
    2,655FG0324Q4LUVJJMS11,Testing AG,

    The directory tree should look like this:

    The test class should look like this:

    AlphaTradeViewTest.kt
    package global.genesis
    import global.genesis.db.util.AbstractDatabaseTest
    import global.genesis.db.util.TestUtil
    import global.genesis.dictionary.GenesisDictionary
    import global.genesis.gen.dao.Trade
    import global.genesis.gen.dao.enums.alpha.trade.Direction
    import global.genesis.gen.view.entity.TradeView
    import global.genesis.gen.view.repository.TradeViewAsyncRepository
    import kotlinx.coroutines.flow.count
    import kotlinx.coroutines.flow.toList
    import kotlinx.coroutines.runBlocking
    import org.joda.time.DateTime
    import org.junit.jupiter.api.Assertions.assertEquals
    import org.junit.jupiter.api.BeforeEach
    import org.junit.jupiter.api.Test
    import javax.inject.Inject

    class AlphaTradeViewTest : AbstractDatabaseTest() {
    @Inject
    lateinit var enhancedTradeViewRepository: TradeViewAsyncRepository

    override fun createMockDictionary(): GenesisDictionary = prodDictionary()

    @BeforeEach
    fun setup() {
    TestUtil.loadData(resolvePath("data/TEST_DATA_VIEW.csv"), rxDb)
    }
    private fun buildTrade(tradeId: String, now: DateTime = DateTime.now()) =
    Trade.builder()
    .setTradeDate(now)
    .setCounterpartyId("2") // COUNTERPARTY_NAME = "Testing AG"
    .setInstrumentId("2") // INSTRUMENT_NAME = "BAR.L"
    .setPrice(12.0)
    .setQuantity(100)
    .setDirection(Direction.BUY)
    .setTradeId(tradeId)
    .build()

    @Test
    fun test_get_single_trade_by_id() = runBlocking {
    val now = DateTime.now()
    val trade = buildTrade("1L", now)
    rxEntityDb.insert(trade).blockingGet()
    val tradeView = enhancedTradeViewRepository.get(TradeView.ById("1"))
    if (tradeView != null) {
    assertEquals("Testing AG", tradeView.counterpartyName)
    assertEquals("FOO.L", tradeView.instrumentName)
    assertEquals(now, tradeView.tradeDate)
    assertEquals(12.0, tradeView.price, 0.0)
    assertEquals((100).toInt(), tradeView.quantity)
    assertEquals(Direction.BUY, tradeView.direction)
    }
    }

    @Test
    fun test_with_single_trade__use_getBulk() = runBlocking {
    val now = DateTime.now()
    val trade = buildTrade("1L", now)
    rxEntityDb.insert(trade).blockingGet()
    val tradeViewList = enhancedTradeViewRepository.getBulk().toList()
    assertEquals(1, tradeViewList.size)
    val tradeView = tradeViewList.first()
    assertEquals("Testing AG", tradeView.counterpartyName)
    assertEquals("BAR.L", tradeView.instrumentName)
    assertEquals(now, tradeView.tradeDate)
    assertEquals(12.0, tradeView.price, 0.0)
    assertEquals((100).toInt(), tradeView.quantity)
    assertEquals(Direction.BUY, tradeView.direction)
    }

    @Test
    fun test_get_multiple_trades() = runBlocking {
    rxEntityDb.insertAll(
    buildTrade("1T"),
    buildTrade("2T"),
    buildTrade("3T"),
    buildTrade("4T"),
    buildTrade("5T"),
    ).blockingGet()
    val count = enhancedTradeViewRepository.getBulk().count()
    assertEquals(5, count)
    }
    }

    You can run the test from IntelliJ by right-clicking on the test class and selecting Run AlphaTradeViewTest or from the command line.

    Adding Testing: AlphaEventHandlerTest

    Now we will add a new automated test for checking the Trade insert method that we created. We are extending the class AbstractGenesisTestSupport to allow proper integration testing. In summary, the new test will:

    So, first, let's do the following:

    1. Add a new test class to the package server/alpha-app/src/test/kotlin/global/genesis/ called AlphaEventHandlerTest.kt.
    2. Add TEST_DATA_EVENTHANDLER.csv to a data folder (server/alpha-app/src/test/resources/data/)
    TEST_DATA_EVENTHANDLER.csv
    #INSTRUMENT
    INSTRUMENT_ID,INSTRUMENT_NAME
    1,FOO.L
    2,BAR.L
    #COUNTERPARTY
    COUNTERPARTY_ID,COUNTERPARTY_LEI,COUNTERPARTY_NAME
    1,335800A8HK6JBITVPA30,Test Ltd
    2,655FG0324Q4LUVJJMS11,Testing AG
    #TRADE
    TRADE_ID,COUNTERPARTY_ID,INSTRUMENT_ID,QUANTITY,PRICE,SYMBOL,DIRECTION,TRADE_DATE,ENTERED_BY,TRADE_STATUS
    00000000001TRSP0,1,1,10,641.927,BRL,BUY,1636987969135,JaneDee,NEW
    00000000002TRSP0,1,1,3,642.927,BRL,SELL,1636987969135,JaneDee,NEW
    00000000003TRSP0,2,2,10,643.927,BRL,BUY,1636987969135,JaneDee,NEW
    00000000004TRSP0,2,2,7,644.927,BRL,SELL,1636987969135,JaneDee,NEW
    00000000005TRSP0,2,2,70,0.0,BRL,SELL,1636987969135,JaneDee,NEW

    The directory tree should like this:

    The test class should look like this:

    AlphaEventHandlerTest.kt
    package global.genesis

    import global.genesis.commons.model.GenesisSet
    import global.genesis.db.DbRecord
    import global.genesis.gen.dao.Trade
    import global.genesis.gen.dao.enums.alpha.trade.Direction
    import global.genesis.gen.dao.enums.alpha.trade.TradeStatus
    import global.genesis.message.core.event.Event
    import global.genesis.message.core.event.EventReply
    import global.genesis.testsupport.AbstractGenesisTestSupport
    import global.genesis.testsupport.GenesisTestConfig
    import kotlinx.coroutines.flow.toList
    import kotlinx.coroutines.runBlocking
    import org.joda.time.DateTime
    import org.junit.jupiter.api.Assertions.*
    import org.junit.jupiter.api.BeforeEach
    import org.junit.jupiter.api.Test

    class AlphaEventHandlerTest : AbstractGenesisTestSupport<GenesisSet>(
    GenesisTestConfig {
    packageName = "global.genesis.eventhandler.pal"
    genesisHome = "/genesisHome/"
    scriptFileName = "alpha-eventhandler.kts"
    parser = { it }
    initialDataFile = "data/TEST_DATA_EVENTHANDLER.csv"
    addAuthCacheOverride("ENTITY_VISIBILITY")
    }
    ) {
    override fun systemDefinition(): Map<String, Any> = mapOf("IS_SCRIPT" to "true")

    @BeforeEach
    fun setUp() {
    authorise("ENTITY_VISIBILITY", "1", "JaneDee")

    val trader = DbRecord.dbRecord("RIGHT_SUMMARY") {
    "USER_NAME" with "JaneDee"
    "RIGHT_CODE" with "INSERT_TRADE"
    }
    rxDb.insert(trader).blockingGet()
    }

    @Test
    fun `test insert trade`(): Unit = runBlocking {
    val message = Event(
    details = Trade {
    counterpartyId = "1"
    instrumentId = "2"
    direction = Direction.BUY
    price = 1.123
    quantity = 1000
    enteredBy = "JohnDoe"
    tradeDate = DateTime.now()
    },
    messageType = "EVENT_TRADE_INSERT",
    userName = "JaneDee"
    )

    val result: EventReply? = messageClient.suspendRequest(message)

    result.assertedCast<EventReply.EventAck>()
    val trades = entityDb.getBulk<Trade>().toList()
    val trade = trades[5]
    assertNotNull(trade)
    assertEquals(6, trades.size)
    assertEquals("1", trade.counterpartyId)
    assertEquals("2", trade.instrumentId)
    assertEquals(TradeStatus.NEW, trade.tradeStatus)
    assertEquals(Direction.BUY, trade.direction)
    assertEquals(1.123, trade.price)
    assertEquals(1000, trade.quantity)
    }
    }

    You can run the test from IntelliJ by right-clicking on the test class and selecting Run AlphaEventHandlerTest or from the command line.

    Find out more about testing

    You can find out more about testing by double-checking our Component testing, Integration testing, and Unit testing pages.

    Additionally, you can see more testing examples by looking at the complete source code of this training available on GitHub.

    Calculated data

    Derived fields are a useful way of providing calculated data, but note that you must only use fields that are in the view.

    derivedField("CONSIDERATION", DOUBLE) {
    withInput(TRADE.QUANTITY, TRADE.PRICE) { QUANTITY, PRICE ->
    QUANTITY * PRICE
    }
    }

    Add this derivedField to your view now. The final view should look like this.

    view("TRADE_VIEW", TRADE) {

    joins {
    joining(COUNTERPARTY) {
    on(TRADE.COUNTERPARTY_ID to COUNTERPARTY { COUNTERPARTY_ID })
    }
    joining(INSTRUMENT) {
    on(TRADE.INSTRUMENT_ID to INSTRUMENT { INSTRUMENT_ID })
    }
    }

    fields {
    TRADE.allFields()

    COUNTERPARTY.COUNTERPARTY_NAME
    INSTRUMENT.INSTRUMENT_NAME
    INSTRUMENT.MARKET_ID withPrefix INSTRUMENT
    INSTRUMENT.CURRENCY_ID withAlias "CURRENCY"

    derivedField("CONSIDERATION", DOUBLE) {
    withInput(TRADE.QUANTITY, TRADE.PRICE) { QUANTITY, PRICE ->
    QUANTITY * PRICE
    }
    }
    }
    }

    Exercise 3.2: derived fields

    ESTIMATED TIME

    20 mins

    Let's add a new derived field in the TRADE_VIEW now. The derived field should display ASSET_CLASS from the INSTRUMENT join. If this field is null or empty, the view should display "UNKNOWN".

    tip

    After changing the files, remember to run build and deploy.

    Consolidators

    Consolidators perform data aggregation and calculations that can either be:

    • real-time - when used as a service
    • on-demand - when used as objects

    Consolidators follow an SQL-like syntax:

    consolidator(TRADE, ORDER) {
    select {
    ORDER {
    sum { price * quantity } into TOTAL_NOTIONAL
    count() into TRADE_COUNT
    }
    }
    groupBy { Order.ById(orderId) }
    }

    In the above example, we aggregate data from the TRADE table into the ORDER table. We group by orderId and we count the number of trades and sum the notional. For further details, see here.

    Some features provided by Consolidators:

    • Type safety
    • Declarative syntax
    • comprehensive built-in logging

    In our case, Consolidators are a good fit for consolidating a position table from trades.

    Define the position-keeping logic in the consolidator

    tip

    To insert new data properly, we need to create 2 additional files called COUNTERPARTY.csv and INSTRUMENT.csv and send it to genesis. this will populate the other two tables you will need.

    Then import the local csv using the Genesis plugin as we saw here.

    Make sure you settled the INSTRUMENT_ID field as not nullable in the TRADE and it is set as an index in the POSITION tables, as the consolidations will use it.

    tables {
    table (name = "TRADE" ...) {
    ...
    field("INSTRUMENT_ID",STRING).notNull()
    ...
    }

    table(name = "POSITION" ...) {
    ...
    field("INSTRUMENT_ID",STRING).notNull().uniqueIndex()
    ...
    }
    ...
    }

    Add the query ALL_POSITIONS in the alpha-dataserver.kts file.

    dataServer {
    ...
    query("ALL_POSITIONS", POSITION)
    }

    When you finish, remember to run generatedao and build and deploy.

    So, let's define a alpha-consolidator.kts file inside server/alpha-app/src/main/genesis/scripts/. This is where you define the consolidator logic.

    The Consolidator is going to increase or decrease the quantity for POSITION records, based on the TRADE table updates. It also needs to calculate the new notional.

    import global.genesis.gen.config.tables.POSITION.NOTIONAL
    import global.genesis.gen.config.tables.POSITION.QUANTITY
    import global.genesis.gen.config.tables.POSITION.VALUE
    import global.genesis.gen.dao.Position
    import global.genesis.gen.dao.enums.Direction

    consolidators {
    config {}
    consolidator("CONSOLIDATE_POSITIONS", TRADE_VIEW, POSITION) {
    config {
    logLevel = DEBUG
    logFunctions = true
    }
    select {
    sum {
    when(direction) {
    Direction.BUY -> when(tradeStatus) {
    TradeStatus.NEW -> quantity
    TradeStatus.ALLOCATED -> quantity
    TradeStatus.CANCELLED -> 0
    }
    Direction.SELL -> when(tradeStatus) {
    TradeStatus.NEW -> -quantity
    TradeStatus.ALLOCATED -> -quantity
    TradeStatus.CANCELLED -> 0
    }
    else -> null
    }
    } into QUANTITY
    sum {
    val quantity = when(direction) {
    Direction.BUY -> quantity
    Direction.SELL -> -quantity
    else -> 0
    }
    quantity * price
    } into VALUE
    }
    onCommit {
    val quantity = output.quantity ?: 0
    output.notional = input.price * quantity
    output.pnl = output.value - output.notional
    }
    groupBy {
    instrumentId
    } into {
    lookup {
    Position.ByInstrumentId(groupId)
    }
    build {
    Position {
    instrumentId = groupId
    quantity = 0
    value = 0.0
    pnl = 0.0
    notional = 0.0
    }
    }
    }
    }
    }

    Update the system files

    Update the processes.xml file

    As Consolidators run on their own process, we need to add a new entry to alpha-processes.xml with the definition of the Consolidator process.

    <processes>
    ...
    <process name="ALPHA_CONSOLIDATOR">
    <groupId>ALPHA</groupId>
    <start>true</start>
    <options>-Xmx256m -DRedirectStreamsToLog=true -DXSD_VALIDATE=false</options>
    <module>genesis-pal-consolidator</module>
    <package>global.genesis.pal.consolidator</package>
    <script>alpha-consolidator.kts</script>
    <description>Consolidates trades to calculate positions</description>
    <loggingLevel>DEBUG,DATADUMP_ON</loggingLevel>
    <language>pal</language>
    </process>
    </processes>
    Update the service-definitions.xml file

    This file lists all the active services for the Positions application. You can see entries have been added automatically when the data server, request server and event handler were generated.

    Add a new entry to alpha-service-definitions.xml with the consolidator details. Remember the port numbers should be free and, ideally, sequential.

    <configuration>
    ...
    <service host="localhost" name="ALPHA_CONSOLIDATOR" port="11002"/>
    </configuration>

    Run build and deploy to verify that the new process works as expected.

    UI configuring

    Let's add a grid in the UI to display the Positions. We could use Entity Management again, but here we will use Grid Pro in @genesislcap/foundation-zero-grid-pro Genesis package presented in Day 2, as this approach offers more flexibility to customise the HTML and CSS.

    First, open the file home.styles.ts and add the code below.

    .split-layout {
    display: flex;
    flex-direction: column;
    flex: 1;
    width: 100%;
    height: 100%;
    }

    .top-layout {
    height: 45%;
    flex-direction: row;
    }

    .positions-card {
    flex: 1;
    margin: calc(var(--design-unit) * 3px);
    }

    .card-title {
    padding: calc(var(--design-unit) * 3px);
    background-color: #22272a;
    font-size: 13px;
    font-weight: bold;
    }

    Finally, go to the file home.template.ts and import the required components. Then, add a constant holding the Position columns, and some <div>s to format the final HTML.

    import {html} from '@genesislcap/web-core';
    import type {Home} from './home';
    import { tradeFormCreateSchema, tradeFormUpdateSchema } from './schemas';

    export const HomeTemplate = html<Home>`
    <rapid-card class="top-layout">
    <entity-management
    resourceName="ALL_TRADES"
    title = "Trades"
    entityLabel="Trades"
    createEvent = "EVENT_TRADE_INSERT"
    updateEvent = "EVENT_TRADE_MODIFY"
    deleteEvent = "EVENT_TRADE_DELETE"
    :columns=${x => x.columns}
    :createFormUiSchema=${() => tradeFormCreateSchema}
    :updateFormUiSchema=${() => tradeFormUpdateSchema}
    ></entity-management>
    </rapid-card>
    <rapid-card class="positions-card top-layout">
    <span class="card-title">Positions</span>
    <rapid-grid-pro>
    <grid-pro-genesis-datasource resource-name="ALL_POSITIONS"></grid-pro-genesis-datasource>
    </rapid-grid-pro>
    </rapid-card>
    `;

    Exercise 3.3: data grids

    ESTIMATED TIME

    15 mins

    Change the Position display to remove the POSITION_ID column, as this field does not have to be in the grid. You can do that by checking our documentation here

    END OF DAY 3

    This is the end of the day 3 of our training. To help your training journey, check out how your application should look at the end of day 3 here.