tl;dr: there’s an extreme need for any tui program to be inherently interactive, and for its code to be built on primitives that translate fundamental relations of data & logic with an interactive ui representation, such as relating partitions to actions i.e. "choosing data to act upon[, one data set at a time]." the tip is to be aware of this blunder, so that you can more clearly decide the extent to which your program will try to re-invent user interaction primitives.
suppose that you want to quickly bang-out a simple script that converts your firefox sqlite cookie database into a netscape cookies.txt file. you’d use a web extension, but none is verified by mozilla, and even the open source ones can’t be trusted unless you build the extension yourself (the author could easily post source code online then use other code in the actual extension!). you don’t want to bother with installing the extension nor grocking the codebase nor auditing the code yourself. you know that the functionality is codeable in a short sql script.
this is an obvious case for bash, if bash weren’t garbage. thus perhaps it’s a good case for dt, but dt’s docs don’t reflect its full functionality, so you haven’t learned it yet. so you fall back to ol' reliable: choose your favorite programming language. the sqlite file is located in ~/.mozilla/firefox/<some profile dir>. usually you’ll have only one default profile, but you might have multiple. aside from that, your situation might want you to choose any given profile for that one occasion. thus your program must present multiple choices, if multiple choices exist, to the user, then accept their input, validate it, and finally once a correct choice is given, proceed.
this is basically simple functionality: just zip & check, but the actual functionality is to print the choices (which should look pretty, such as a tabular format, if many choices are displayed beyond the viewport’s width or height), and print a message if an invalid choice is given.
<enumerated> "choose:" print dup [ CHAR: . swap "%d%c %s\n" printf ] assoc-each
f swap '[ readln string>number _ at [ nip f ] [ "invalid" print t ] if* ] loopit’s not horrible, but this functionality should come built-in to a scripting language designed for general interaction with a computer system i.e. a user’s default shell which interacts with the computer: filesystem, ports, os state, etc. for an operating system to not provide an api for user interaction is a design blunder! rather, posix shells, which of course come with all \*nix oses, as is necessary to interact to the machine at all, are poorly designed because they aren’t designed to be interactive well. we don’t quite need curses, though that would be nice, provided that specifying layout flows is easy; instead, we can use some few display primitives, such as tables (whose layouts already commonly fill a terminal’s viewport). just as we don’t have to specify dimensions for printing a table, so we don’t need to specify them when doing a more-curses-like ui. also, just as one can use a tiling window manager with only the keyboard quite well, so can one manipulate the layout of a terminal interface using only the keyboard. ideally, the window manager would be one with a terminal emulator so that the programmer’s code would specify multiple elements, and their dimentions/layout/flow would be determined by the window manager and specified interactively by the user (again, as is already the case for tiling wm’s). the design mistake for text shells was to base everything on text streams alone—good for programming (but even there, overly simplistic) but bad for computer use (which is inherently interactive); and gui’s design mistake was not exactly to separate programs into windows, but rather to not enable multiple windows to affect their layout altogether; gui programs can communicate, but there’s no mechanism for the window manager to change multiple windows' displays based on their state!
certainly you may say "if the user selects an invalid choice, then the suffer for their own stupidity", but there are valid times when mere absentmindedness, or getting used to a new keyboard layout, or merely typo’ing. then again, these could result in the user entering a valid but undesired choice, wherein anyone would assume the program to accept then immediately act upon it…. anyway, having the program crash because of a bad input is stupid simply because generally any program losing state is stupid, except for hardware failures which are outside the scope of programming. it’s not just losing state, though: it’s being able to not undo! how much code would we need to write to enable the user to stop execution, change their choice, then resume from that point? that’s unheard of for repls, but commonplace anywhere else in life!
for all the thought about such a common situation, i haven’t even gotten to the actual core functionality: converting sqlite to text! a program’s central functionality is our desire, but it cannot work unless the underlying ux is good!
unix ipc is hardly interactive; more aptly, it’s para-active! programs pass data on to another program, both unaware of each other, each aware only of the transited data. passing a baton in a relay race shouldn’t be called "interactive" if a therapy session or other dialogue is also so called. at least, "interactive" should mandate the use of a duplex stream! certainly piping small programs in a decently interactive repl (terminal) is better than larger programs that can’t interact easily, but repls are pathetic compared to text editors! the aren’t as interactive, they don’t have a good "undo", their multiline editing is poor, and they support no keybinds or any of the great functionality customization that we expect from text editors!
but the most significant difference between a repl and text editor is whether they have primitives about interactivity with the user! all of a text editor’s primitives are about moving a cursor or modifying the state! repls commonly have readline(1) functionality, but there’s no copy & paste, no marks, registers. you can move the cursor, but you’re hardly modifying state! a good ux is one that displays state and allows the user to modify any subset of said state, one or multiple commands at a time, with undo, where the commands themselves can be interactively edited well, too.
TODO: enumurate a proven-total set of primitive relations and their ui representations. this will almost certainly exactly be that set editor that i wrote about…somewhere. after all, all computing is relating subsets to actions.
of course, then there’s the issue of "how do i get my proglang to interact with the os-provided api? is there a wrapper…?" etc. this is the 2nd reason why getting a computer to do anything is total ass: programming languages. to have programming be specified by a language was just stupid. again, it should’ve been "here’s the computer’s abstract design, the hardware that implements it and how, and a system for interacting with the computer['s state]." the 3rd item is what the programming language is to satisfy, but i’m one of many who’re unsatisfied with it. and why do we have proglangs? because we had typewriters, which became keyboards, then unix with its piping of character streams and its repl shell. the computer is a machine capable far beyond a typewriter, and that has always shown, but damn, we can still move much further beyond the trappings of typewriters! …ah, i’ve found myself into the same rant as i ranted…hell, a year ago? see humane-computing.adoc. yeah, there’s no avoiding this design flaw!
like, isn’t it stupid to code in a text editor, which has all this text-specific behavior, such as send of word", "surround in quotes", "split on delimiter", etc, but it’s all about text instead of code—for the editor to know the semantics corresponding to the syntax—and then to, given all that functionality, pass the produced text to a separate program that has to parse it all from scratch? why have a separate text editor and text-to-program parser? why not just one program editor?! if you can write a parser, you can write an editor, at least provided a sensible ui library. and don’t respond, "one editor enables one to code in many languages" because to the extent that the editor isn’t tailored to any language, it’s less good, and to the extent that it accounts for multiple languages, it just makes me wonder why so many languages have such similar syntax. like what’s the point of making a new language, then? and if all languages are basically algol (c/++, java, python, and every other procedural, applicative language with common function syntax and lexical scoping), then "multiple languages" is mostly a lie anyway! if there’s one thing that i can say after 16 years of coding, it’s that all the languages yet suck—just some less than others—and there are basically 5 languages: ml, algol, apl, prolog, & forth. some few are combinations, e.g. rust being ml & algol. yes, erlang is notable for its actor model, and pony for its termination guarantees, go for its concurrency, and rust for its memory safety, and any one of these languages can be used for its one thing that it does better than others, but that’s not much, and couple a particular syntax with semantics is uncessary & foolish; an execution model may come provided with a syntax, but people should release execution models alone, allowing users to easily provide their own syntaxes. and "just make your own syntax compile to the provided one" is dumb; most languages' syntax is more complex than necessary, so a simpler syntax (e.g. bytecode) would be easier to programmatically produce. but this being said, if it’s simpler, why not just have it be used instead of the bloated syntax?! apparently some people just prefer bloat…? *shrug\* whatever. may they be free to choose it just as i be free to choose else. on principle, though, simply, functionality does not have an inherent appearance, so don’t give it one!
when i think about this all, then i look at how developing propgrams has been going for me, i wonder why programming is ever difficult at all. it’s coverage, not just of values passed to fns, but of situations e.g. what if the socket breaks? should i immediately reconnect or log an error then die? i guess it’s the same old stuff as ever: initial values &al numeric concerns…an interesting open question is what set of primitives can express all programs, esp when separated into numeric (init vals, updates thereto, coverage) vs orchestral (e.g. sequencing i/o actions) concerns.
i:/2 (find-last) is just a convenient form of what’s basically linear search under reverse. i./2 is useful doing a binary search but when you don’t want to do a binary serach for some reason, e.g. having a list of duples but wanting to test only on one of the elements. find-last runs in O(n):
"abaabc" { 10 11 15 18 25 49 } zip [ first2 [ CHAR: b = ] dip 17 <= and ] find-last nip{ 98 11 }. had i only find, and neither find-last nor reverse, i’d have to keep the latest-yet-encountered element of the "b" class, then test the current value to see whether it’s greater than 17, and if so, then exit the loop:
"bbaabc" { 10 11 15 18 25 49 } zip
f swap ! init state: latest-yet-encountered elt of the "b" class
[ dup first2
{ { [ 17 > ] [ 2drop t ] }
{ [ CHAR: b = ] [ nip f ] }
[ drop f ]
} cond
] find 2dropgranted, if i accept a filter, then of course i’d just use binary search afterward, which is very simple code:
17 "abaabc" { 10 11 15 18 25 49 } zip [ first CHAR: b = ] [ second ] filter-map natural-search nipwhich correctly gives 11. were i to filter then linear search:
"abaabc" { 10 11 15 18 25 49 } zip [ first CHAR: b = ] [ second ] filter-map
[ [ [ 17 <= not ] find drop ] keep length or 1 - ] keep nthwhich sucks b/c i must find the earliest element that fails to match the predicate, then get its predecessor.
all this to say: i:/2 has its place; it’s not simply an alternative to i.@|.. reversing a list still retains the same information which requires us to offset the found index by 1.
lessons learned:
-
binary search is good, and is particularly afforded when we store data separately, relating them by indices only whenever necessary
-
a stateful traversal was expressed statelessly by reversing the traversal direction; or
-
the need to offset a stateless traversal’s output was avoided by reversing the traversal direction
granted, had the list been ordered descending, then i’d’ve just used find instead of find-last in the first place.
yet, even all this said, were i to run this search on an ascending sequence (17, then 19, then 20, 21, 26,…), then i cannot use i:, since the found index will increase, not decrease! still, the answer to this is to use bins, namely where its right argument is recognized to be an ascending array. were bins to operate on each of its right items independently, it would be very inefficient!
perhaps best would be to grade & sort the right argument, use the specialized search, then use that same sorting grade backwards to match the result set to the original pre-sorted sequence. i see this as being a similar advantage to linear serach under reverse: we use "under" to normalize data so that it’s suseptible to performant algorithms. i wonder how few algorithms we need if we use them together like this. how efficient can it be to normalize data so that it permits specialized algorithms, all the while retaining association with the arbitrarily-ordered/shaped data? indeed, all data may be considered as an indexed set, and sets aren’t ordered, which means that they’re equivalent under ordering, so if we can exploit this for computation’s sake, then why not. at every step between computations, each datum input can be associated with any output set: a→b ⇒ b→a; a→[b] ⇒ [b]→a (e.g. (],.i.)); and [a]→b ⇒ b→[a] (e.g. (],.+/@:])@i.). this all is possible if we have built-in support for indices. obviously duplicate values out of context cannot uniquely be mapped to a preimage, but they of course can if we retain indices, since for every domain, indices are bijective with elements!
-
x is 0 or length:
x length mod 0 =; i say "length" here b/c this method assumes that x must be less than2*length
NB. i’m using factor here only b/c: 1. it’s one of my preferred lang for expressing algorithms; 2. its uint-4 type is specifically for simd.
: I. ( n -- idx )
[ > ] curry
{ ... } ! a monotonically increasing seq
[ swap find drop ] keep length or ;this is a linear search rather than a binary one, but for such small sizes, linear is probably faster. only for such small sizes does the following optimization exist anyway:
: I. ( n -- idx )
4 swap <repetition> >uint-4
uint-4{ ... } ! you may need to pad right side with max uint4 value, 0xffffffff
v>= vcount ;the benefit is the ability to use simd when there’s a simd instruction for "count" but not for "first set index". the simd version obviosuly limits the traversal space’s length, but uses only 2 cycles and no looping.
the actual version that i used in production code was:
: mins-since-00 ( millis -- mins ) 86400000 rem 60000 /i ; inline
: millis>session ( millis -- s∈[0,3] ) mins-since-00 4 swap <repetition> >uint-4 uint-4{ 420 570 960 0xffffffff } v>= vcount ; inlinewhich converts a millisecond timestamp into one of 4 stock market sessions. the uint-4 values are minutes since the start of the day (i.e. midnight): 07:00, 09:30, 20:00, and, lastly, a number guaranteed to be greater than all others (actually, anything greater than or equal to 1440 would do, here); uint-4 mandates that i specify exactly 4 values, so it’s a dummy. this is equivalent to j code 0 420 570 960&I..
|
Tip
|
i can store sessions as 23 bits: 21 msb’s for jdn which has a value range of [1029/09/15,6771/07/06], and 2 lsb’s for the session number. such values' order equals temporal order. |
-
given a unix timestamp, to truncate all seconds since the start of the day:
86400000 [ /i ] keep *where/iis integer division i.e. division with truncation and 86400000 is the number of seconds in a day. -
rounding is the same as truncation except that instead of division with truncation, we divide to produce the dividend and remainder, then add 1 to the dividend if the remainder is at least half of the divisor:
86400000 [ /mod ] keep [ 2/ >= 1 0 ? + ] keep *where2/is division by two accomplished by simply shifting an integer down one bit. in languages where 0 is falsy and all other numbers are truthy,1 0 ?is not needed.-
if you’re working with reals (floats or rationals), then you can add 1/2 then take the floor e.g.
86400000 [ / 1/2 + floor ] keep *.
-
TODO: merge with ramble in incremental.adoc
of course we can inline what would otherwise be a lambda. (TODO: give example here) however, i came across a pattern where, rather than passing a referentially transparent function to a higher-order function and having the h.o.f. use its output then e.g. collect it into an output data structure, we define the function to accept a pointer to the output, then have the function mutate the output, and call this function from within a loop.
this github gist for incremental moving averages (which i’ve re-expressed much more tersely here) in c demonstrates that we can pass an array to a moving average function, and use it in a loop, rather than passing the array & function-as-a-lambda to a loop. in this example, said function is movingAvg, buf is a loop state, and sum is a pointer to an output value. rather than the output being collected, it’s put into stdout.
#include <stdio.h>
#define k 5
int movingAvg(int*buf,int*sum,int j,int x){*sum=*sum-buf[j]+x;buf[j]=x;return *sum/k;}
int main(int argc, char *argv[]) {
int xs[] = {50, 10, 20, 18, 20, 100, 18, 10, 13, 500, 50, 40, 10};
int buf[k] = {0}; // effectively a ring buffer
for(int i=0,sum=0;i<sizeof(xs)/sizeof(int);i++){
printf("new avg: %d\n",movingAvg(buf,&sum,i%k,xs[i]));
} return 0;
}actually, obviously this corresponds to inlining: whatever mutations would be inlined can be passed by reference to a separate function. any functionality (mutation & arithmetic) can be factored-out into a separate function and all concerned data may be passed as parameters thereto.