Next: , Previous: , Up: Top  


11 I/O and Interface

In order to support fast data analysis, k9 supports fast input and output (I/O). If you have the habit of making tea while a huge csv file loads, you might have to opt to take a few sips from a glass of water instead.

k9 is also able to interface with other languages so one doesn’t need to translate everything you’ve already written in those other languages to k9.

 i/o(*enterprise)
 0: r/w line
 1: r/w char
*2: r/w data
*3: k-ipc set    
*4: https get
*5: ffi:`f!"ifsIF"

jk
csv

11.1 Example of Data I/O

Let’s begin with a simple example to show how to read data from a csv file. We’ll generate a small table and save it with and without headers (wHeader.csv and woHeader.csv respectively). Then we’ll read it back in, both specifying types and with the csv reader. Types are specifed by one letter and the full list can be found in Atom Types.

 t:[[]a:3 2 1;b:1. 2. 4.;c:`x`y`z];t / generate table
a b  c
- -- -
3 1. x
2 2. y
1 4. z

 "wHeader.csv"1:`csv t               / save with column headers
wHeader.csv

 `csv?1:"wHeader.csv"                / default reader
a b  c
- -- -
3 1. x
2 2. y
1 4. z

 "woHeader.csv"1:1_`csv t            / save without col headers
woHeader.csv

 (",";"ifn")1:"wHeader.csv"          / separator and types
a b  c
- -- -
3 1. x
2 2. y
1 4. z

 (`e`f`g;",";"ifn")0:"woHeader.csv"  / names, separator and types
e f  g
- -- -
3 1. x
2 2. y
1 4. z

11.2 read csv ⇒ csv[x;y]

Enterprise only

Convert y from csv format using type(s) x. Before using csv one must load the csv.so (linux) or csv.dylib (mac) shared library. Types are integer (I), float (F), string (S), date (D), and time (T).

 cjF:*"./csv.dylib"5:(,"_Z2ckPcS_")!," ss"; csv:{cjF[x;y]}
 0:"some.csv"
x,y,z       
1,1.,blue   
2,0.1,red   
3,0.01,green

 csv["IFS";1:"some.csv"]  / y is not the file name but the file contents
x y    z    
- ---- -----
1 1    blue 
2 0.1  red  
3 0.01 green

11.3 read json ⇒ jk x

Enterprise only

Convert x from json format. Before using jk one must load the json.so (linux) or json.dylib (mac) shared library

 jk:*"./json.dylib"5:(`$"_Z2jkPc")!" s" / load shared library
 jk"{\"a\":[0.0,1.0,2.0], \"b\":true}"  / convert from json string
a|0 1 2.
b|1     

 jk@1:"ab.json"                         / convert from file input
a|0 1 2.
b|1     

11.4 write line ⇒ x 0:y

Output to x the list of strings in y. y must be a list of strings. If y is a single string then convert to list via list.

 ""0:("blue";"red")      / "" represents stdout
blue
red

 "file.txt" 0: ("blue"; "red") / write to file, one line per element

11.5 read line ⇒ 0:x

Read from file x.

 0:"some.csv"
a,b 
1,3.
2,4.

11.6 write byte ⇒ x 1:y

Output to file x the list of chars in y as bytes.

 "some.txt"1:"0123ABab"
"some.txt"

 1:"some.txt"
"0123ABab"

 "some.csv" 1:`csv [[]a:1 2;b:3. 4.]
"some.csv"

 1: "some.csv"
"a,b\n1,3.00\n2,4.00\n"

 `csv?1:"some.csv"
a b
- -
1 3
2 4

11.7 read byte ⇒ 1:x

Read from byte data from file x. See w byte for an example to write and then read byte data.

11.8 read data ⇒ 2: x

Enterprise Only

Load file, eg. csv or from a (x 2: y) save. For the latter, one can find a “save then load” example in the next section.

 2:`t.csv
s    t        e p  z   
---- -------- - -- ----
AABL 09:30:00 D 11 4379
AABL 09:30:00 B 40 3950

 2:`r                / read from file
a          b          c          d          e         
---------- ---------- ---------- ---------- ----------
0.5366064  0.8250996  0.8978589  0.4895149  0.6811532 
0.1653467  0.05017282 0.4831432  0.4657975  0.4434603 
0.08842649 0.8885677  0.23108    0.3336785  0.6270692 
..

11.9 write data ⇒ x 2: y

Enterprise only

Save to file x non-atomic data y (e.g., lists, dictionaries, or tables).

This example saves 100 million 8-byte doubles to file. The session is then closed and a fresh session reads in the file. Both the write (420 ms) and compute statistics from the file have impressive speeds (146 ms) given the file size (800 MB).

 n:_1e8
 r:+`a`b`c`d`e!5^n rand 1.;r
 `r 2:r              / write to file 

Start new session.

 \t r:2:`r;select avg a,sum b, max c, min d, sum e from r
148

11.10 IPC

k9 has the ability to comunicate between separate k9 instances via inter-process communication (IPC). These processes can be on the same or separate machines. A user will start multiple instances specifying the port that the k9 session will use, opening a handle to that port, and then running remote commands via the handle and 3: or 4:.

11.11 k-ipc ⇒ 3:x

Open a connection to port x and return a handle. If the port has been forwarded from another machine (eg. ssh -L 1280:server.com:1280 laptop.com) then this handle allows remote execution.

 / before running this session start another k session in another
 / terminal window specifying that port 1251 should be used
 / k -p 1251
 h:3:1251 / create handle to port 1251 and save to h
 h        / h is an integer
4

11.12 set ⇒ x 3:y

Exectute string y using handle x. Handle x should have already been created using k-ipc. set is asynchronous, thus once the command is sent the session immediately returns to the user.

 h:3:1251               / create handle to port 1251
 h 3:"a:12"             / set a to 12 on remote session
 h 3:"n:_1e8;b:1?n?1."  / set b to a random number on remote

11.13 get ⇒ x 4:y

Execute string y using handle x. Handle x should have already been created using k-ipc. get is synchronous, thus on the command is sent the session waits until there result is ready.

 h:3:1251               / create handle to port 1251
 h 3:"a:12"             / set a to 12 on remote session
 h 3:"n:_1e8;b:1?n?1."  / set b to a random number on remote
 h 4:"a"
12

 h 4:"b"
,0.07820029

 a:"local"              / set a locally to a string value
 a
"local"

4:"a"
12

Next: , Previous: , Up: Top