As you have hopefully seen from some of the examples included with Sloup,
a transaction with Sloup consists of a sequence of command lines sent from
the desktop. The simplest sequence consists of:
- a soup name
- a entry specification: the order and type of data values
- the data itself: a line per record
- end of transaction
There are several variations on this: other commands/lines that can be
included; and a "line" can consist of several lines, but before getting into
that complexity, let's look at a sample transaction in slow motion to see
how Sloup's status messages reflect where it thinks it is in this sequence.
- you tap Connect in Sloup (and open connection from desktop)
- Sloup displays "Connected", "Waiting for Soup Name"
- you send a soup name
- if the soup exists (or if you've asked to create a soup) and it is
successful, Sloup displays "Waiting for EntrySpec". If it is unsuccessful,
Sloup returns to "Waiting for Soup Name"
- you send an entry specification (entryspec)
- if the entryspec compiles successfully, you've just created a
NewtonScript frame object that describes the data (order and types) and
Sloup displays "Waiting for Data"; if unsuccessful, Sloup returns to
"Waiting for Soup Name"
- you send a data line
- if the data is correct (number of fields, types of values), Sloup should
successfully map and convert the data using the entrySpec, store it in the
soup, and increment the Entries: count; if unsuccessful, you may see an
error notification. Sloup continues to wait for data.
- you send "BYE!"
- Sloup finishes with the current soup, plays a sound (if selected), and
returns to "Waiting for Soup Name"
Here is a preview of some possible variations in the above example of a
Sloup file/transaction. These will be explained in more detail later.
- soupName
- besides an existing soup like Notes or Names or one that you or an
application created earlier, there can be several special names and
commands. For example, you can use "Sloup" to change
global settings such as keyboard mapping or field delimiters; DUMP! lists
names of soups; EVAL! evaluates and prints a
NewtonScript expression; Package (on NOS 2.x) sets up
for import/export a single package; you can create a soup by appending the
soup name with !, followed by an array of index definitions.
- entrySpec
- this is a format frame. in addition to defining the format for incoming
data, an entrySpec can be used to define global settings (for the Sloup
"soup"), to define output format for exporting (DUMP!)
data, and to limit specific data to DUMP (a very limited, non-general query
capability)
- data
- sometimes an initial data line may actually be a command, such as ERASE! to erase entries, or REPLACE! to define how to replace (rather than just add
entries). For Notes, a line is raw text concatenated with other lines (until
----- is encountered) to create a single entry; empty paragraph entries are
ignored. For other soups, a line is a sequence of tab-delimited fields; a
single line represents an entry, unless a line continuation \ is used; empty
lines are ignored.
- BYE!
- a DUMP! might occur here to cause entries to be exported (according to
earlier entrySpec); a BYE! might be followed by an array of definitions for
Soup Indexes (created at end of process, rather
than at beginning after soupName), e.g.,
MySoup![]
{....}
...
BYE![{structure: 'slot, type:...}
This allows the data entries to processed faster; however, when indexes
are added at the end, the total time is greater (since entries have to be
accessed again). Sloup only adds the indexes if there are none present,
e.g., it assumes you have removed the soup earlier, or are using the same
existing indexes. (To add just some indexes to some stores involves too much
code that relies on undocumented features of how soups are represented). I
would generally recommend that you add indexes first, but now you have a
choice.
Since there are some differences between import and export, and between
Notes and other soups, here is a summary of the four basic kinds of transfer
supported by Sloup:
- Import text to Newton
- Export text to desktop
- Import data to Newton
- Export data to desktop
Here is a simple example of importing text into the Business folder.
Notes
{labels: 'Business, viewFont: 10241}
text
a second line in same paragraph
-----
another para
-----
-----
BYE!
Notes is the soup name for the notepad; 2.x users can also use Newtworks
to transfer plain text documents to Newtworks.
The entrySpec for Notes/Newtworks is used to specify several default values.
- labels
- labels is a symbol corresponding to a folder destination, e.g.,
'Personal, 'Business, or nil or 'nil (for Unfiled). If the label contains
non-alphanumeric characters, you need to wrap the symbol with vertical bars,
e.g., '|two words!|. If you include a symbol for labels slot that does not
(yet) exist, be sure to Edit Folders later on your Newton to add it --
otherwise, the only way you'll see these notes is via All Notes
- viewFont
- If viewFont is not specified, then the Note is created with the current
default font set by the user in Styles. If viewFont is specified, it should
be a font specification, e.g., a coded integer like 10241 (userFont10) or
font frame description like {family: 'espy, face: 0, size: 9} -- see Newt or
NTK documentation. I have included viewFont: 10241 in the .nwt files to be
consistent with Newt's default
- height
- If height is not specified, then Sloup creates each Notepad entry with
its calculated height. You can specify a fixed height, e.g., {labels:
'Personal, height: 100}. If the Notepad clips a long para, hold down on the
note separator bar in the NotePad for several seconds, then drag it
downwards (it disappears) and then let go -- the separator should jump to
the end
- title
- NOS 2.x: Sloup uses the first line of the text as a separate title for
the Note. Note: NewtDevEnv currently expects the header line to be the first
line of the text note (and not stored separately in the title); if for some
reason you want to specify a title for Newt files for 2.x, you should add a
separate title line (possibly a duplicate of the header line)
- class
- NOS 2.x: 'list (outline), 'checkList or 'paperroll (default). For lists,
tabs in the data lines are used to indicate level.
Notes entrySpec import examples:
- {labels: 'Personal} // uses current default font (Styles)
- {labels: nil, viewFont: {family: 'espy, face: 0, size: 9}, height: 500}
- {} // Unfiled folder with default font and calculated height
- {labels: nil} // same as {}
- {labels: 'nil} // same as {}
If you know what you are doing (and have backed things up first), you can
insert an ERASE! command after the entrySpec. This erases all entries from
the specified soup; or if you've specified labels in your frame spec (for
Names or Notes), it erases just the entries in that folder; if a field has
labels, you can specify the symbol _all to erase entries in all folders,
e.g., {labels: '_all}.
ERASE! can be handy especially if want to get rid of
many existing entries before adding new information (also see REPLACE! for selective replacement). Warning: this does
not give you a dialog box asking to Confirm erasing of
entries!
For example, if you edited Newt sources for your project on the desktop,
and wanted to replace these completely on your Newton, you could do
something like:
Notes
{labels: 'Personal, viewFont: 10241}
ERASE!
myApp
{_proto: protoApp,
}
-----
myApp+aButton
{_proto: protoTextButton,
viewBounds: RelBounds(10,20,50,15),
}
-----
BYE!
Although a soup is like a database and may have an index that treats a
particular slot as a "primary key", Sloup does no checking to eliminate
duplicates. So, it always adds new entries. Instead of erasing an entire
folder and then storing entries, you could instead specify a slot to
identify an entry; if an entry with the same name already exists, the new
entry would replace the entry. You can also remove individual entries with a
soup utility.
For Notes, title is the only slot supported, so REPLACE!title would
replace any note that has the same title -- this works on 1.x by using the
first line of the text as the "title". For Newt, the first line of a note is
typically a unique identifier, e.g., "MyApp+button1"; for Newt's Cape, the
first line could be a comment indicating the filename, e.g.,
<!--foo.htm-->
Note: all entries are added to the default store, e.g., on your memory
card if "Store new items on card" is checked. If you "replace" an existing
entry, the old entry is actually removed from its existing store, and a new
entry is created on the default store.
You can Export text to desktop and
Export data to desktop by using DUMP!.
You can follow DUMP immediately with a frame, i.e., DUMP!{...}) to specify
parameters related to delay, print function, field, record and soup (eof)
delimiters. Final field always is followed by a field delimiter (previously,
only record delimiter). Here are the slots you can specify (along with
defaults):
- delay
- number of seconds to pause. default: 0 (no delay); handy for setting up a log/capture file
- printFunction
- NewtonScript function definition with single parameter. default: Functions.SPrintObject
- fieldDelimiter
- string to output after each field. default: tab ("\t")
- recordDelimiter
- string to output after each record. default: none
- soupDelimiter
- string to output after all records. default: "BYE!"
- noAddDelimCR
- default: nil. controls whether to add crlf after recordDelimiter or
soupDelimiter (default or user specified) -- DUMP!{..., noAddDelimCR: true}
means do not add cr. (default adds it)
Each note is separated by a line of at least 5 dashes, i.e., -----. The
last note separator is optional: BYE! will also terminate the last Note.
Empty notes are not saved to the Notepad.
For NOS 2.x systems, you can import text into outlines and checkLists
using class. Each line is an entry; tabs indicate
level. For example, this would add two checkLists to the Unfiled folder:
Notes
{labels: nil, class: 'checkList}
a checklist in Unfiled
second level
a next level item
-----
another checklist
indented1
indented2
another main entry
indented1
BYE!
Add two lists to 2.x Business folder (lists are called outlines in the New picker):
Notes
{labels: 'Business, class: 'list, viewFont: {family: 'casual, face: 0, size: 10}}
an outline in Business
indented item
another indented item
-----
another outline
indented1
indented2
another main entry
indented1
BYE!
For dumping (exporting) text to the desktop from Notes (or Newtworks for 2.x users),
you can specify a folder, and several fields:
- labels
- if not provided (or '_all), Sloup dumps all folders; otherwise, it's a
symbol for an existing folder (or nil or 'nil for Unfiled)
- title
- if present (type: "string"), Sloup outputs the title of the Note (for
NOS 2.x) or a copy of the first line (1.x)
- _modTime
- if present (type: "dateTime"), Sloup outputs the last modified date
using the date format specified
- class
- if not present, Sloup dumps all notes, including lists. If present (NOS
2.x), Sloup restricts to just that class of note,i.e., 'paperRoll, 'list,
'checkList
Note about field order: Sloup does not output class, and it defers title
until just before the text; otherwise, fields (labels and _modTime) should
appear in the order specified. Currently, entries are dumped from all
mounted stores (internal, card,...).
DUMP! all Notes in Business folder
Notes
{labels: 'Business}
DUMP!
For NOS 2.x systems, that specification would have included all classes
of Notes. You can also select just lists (outlines) or checkLists.
Dump just Unfiled checkLists:
Notes
{labels: nil, class: 'checkList}
DUMP!
Dump just plain notes (no checkLists or outlines) from all folders, along
with titles and modification dates:
Notes
{class: 'paperRoll, title: "string", _modTime: "dateTime"}
DUMP!
Dump just Business outlines (i.e., lists):
Notes
{labels: 'Business, class: 'list}
DUMP!
Importing general data is more complicated than for Notes: there can be
many fields and different data types, depending on the soup --these fields
and types may or may not be documented. Each data entry is represented as
single line of tab-delimited text (unless there are line continuations). For
example, see CasioNam.tab. Blank lines are ignored.
Sloup assumes that no more than approximately 20 fields are embedded at
one level in one frame (you could have more than 20 fields total if you have
nested frames or arrays); the order of values in a line must correspond to
the order of slots in the frame spec. For small (<20) number of slots, the
frame spec maintains a linear order when it is compiled; however, for larger
number of slots this ordering is not preserved (I have no plans to redesign
Sloup to avoid this limitation).
Since exporting uses the same entrySpecs, I will discuss these and
provide examples a little later.
Assuming you have a valid entrySpec, export (DUMP!)
is quite simple. The following example works if you have already created
"testSoup" (if not, go to the later Soup Indexes
section; then come back)
TestSoup
{a: "int", b: "string"}
DUMP!
For example, see CasioNam.dmp. Entries are written to terminal emulator
as tab-delimited lines in frame specification order. (If possible, you
should have your terminal emulator preserve tabs, wrap lines (rather than
overwrite characters), and save/capture to a log file). Any data fields with
text values that contain newlines will be transmitted as is -- you may need
to re-edit results into a true tab-delimited text format, or modify field
delimiters.
If you do not know the fields and value types for a soup, you can specify
an empty frame {} as the entrySpec. In this case, Sloup will dump an
pseudo-entrySpec based on the first soup entry, and then dump all remaining
entries with respect to this entry. (This works only if levels of the soup
structure are less than 20 in length, and subsequent entries are similar in
structure to the first entry).
TestSoup
{}
DUMP!
Although you can use Sloup to discover soup structure, I would recommend
that you use documentation wherever possible (e.g., additional Sloup docs
for built-in soups like Notes and Names), or soup utilities (e.g., StewPot)
or NTK Inspector. And of course, backup your Newton to avoid possible
damage.
Special "soups"
There are several special names that can occur in the soupname position
(first line of a transaction) and are handled specially.
- DUMP!
- If you do not know the names of the soups on your Newton, you can use
DUMP! in place of the soupname and Sloup will list names of soups on all
stores (there may be redundancy)
- EVAL!
- EVAL! can be used to evaluate and print arbitrary NewtonScript
expressions, e.g., EVAL!3+4. You will have better print results and error
reporting if NewtDevEnv is installed.
- Package
- On NOS 2.x, you can list packages, or import or export an individual
package. This does not actually access the "package soup" directly but uses
a different API. Import/export examples later under binary:packageEntry. To list
package names:
Package
{}
DUMP!
- Sloup
- If the first line is Sloup (or "Slurpee" in older scripts), the entrySpec is used to specify global
settings; it returns to waiting for a soup name -- there is no data. For
example, if you want to change some global settings and then restore
defaults, here's how it might appear:
Sloup
{fieldDelimiter: ",", stripQuotes: true,...}
Names
{....}
...
BYE!
Sloup
{}
Or if you had earlier made global changes earlier and wanted to ensure
that defaults were used:
Sloup
{}
...
Here are the global fields you can set in a Sloup entry:
- fieldDelimiter
- default: tab, i.e., "\t". string with new field delimiter, e.g., ","
- stripQuotes
- default: nil. if true, strip any beginning and ending " from each data field
- translationTable
- default: nil. specifies an array of pairs of strings -- the first is a
pattern that you want to replace in String values, the second is the
substitution. This can be useful for international users who want to specify
special characters but whose character encodings are different from the
Newton's. These can be single unicode characters (or anything). These
substitutions are applied starting at the beginning of the list. for
example, translationTable: ["\u2022", "\u00D1"], would substitute an "enye"
character for the first one. Currently not used for DUMP. Note:
translationTable is applied to all lines, so this affects Notes as well as
tab-delimited data (and may affect other commands, specs)
- totalRecords
- default: nil. a number. If you know in advance how many records you
have, include a number, e.g., totalRecords: 500. Then, "Entries:" will
display 1/500, 2/500, etc.; a gauge also appears
- convertUnicode
- default: nil. controls whether to convert \u unicode sequences.Use \u to
start a sequence (must be multiples of 4); sequence ends with \u or end of
each line. This might be more convenient that creating a translationTable.
One user is already using this to transmit Chinese characters to the Newton.
For example,
Sloup
{convertUnicode: true}
Notes
{labels: nil}
first line
upsidedown question mark \u00BF\u embedded
enye at end of line \u00F1
\u00A700A52126\u -- a few in a row
in lowercase: copyright \u00a9\u yen \u00a5\u
BYE!
Sloup
{}
- debug
- default: nil. if true, Sloup will print the unicode (4 char hexadecimal) code for each keystroke. This can be handy to determine what code is actually getting to the Newton. When specifying a code, you can prefix this unicode character with \u
- clearKey
- Depending on your terminal emulator and operating system, you might want to remap this or other characters. default: "\u001B" (esc). If you type the Clear key, the current field is cleared (with a poof sound) -- unless the current field is Newt's source editor, in which case, it Reverts to the last Saved entry. After a Clear in the Notepad, you need to tap to reselect the view
- evalKey
- default: "\u001A" (ctrl-z). Evaluates the current selection or
field (in Newt, in the Notepad, or any input field) as a NewtonScript
expression. If Newt is installed, Sloup uses Newt's read/eval/print
capability for the current expression; otherwise, Sloup does a simpler
compile and print (only of immediate value results: strings, numbers,
characters). Any errors messages also appear in the terminal window. (Newt,
if present, decodes some of the common error codes). After an Eval, you'll
hear a "plunk" sound. If the current field is Newt' source editor, the
current object or method is Saved (it's checked/compiled) Some expressions
to try:
3
3/ //syntax err
3/2
3/x //undef var
3/0 //div by 0
[2,4,6,8,10][0]
"Hello," && userConfiguration.name
{person: {lastName: "Smith", firstName: "John"}}.person.lastName
GetRoot():Notify(3,"AN ERROR!", "(not)")
- enterKey
- default: "\u0003" (ctrl-c). same behavior as evalKey --
provided as an alternate key binding
- closeKey
- default: "\u0017" (ctrl-w). closes 'viewFrontMost. This might
be useful for closing Eval Log (after errors in Newt) or your application,
though you'll probably want to set Newt's 'closeConfirm preference to avoid
inadvertently closing Newt. This does GetView('viewFrontMost):close(), which
may not always do what you want/expect. viewFrontMost means the frontmost
view that has vApplication set. Another option might be viewFrontMostApp
(same but ignores views with vFloating set). viewFrontMostKey might be a
possibility, but it typically refers to the currently selected view, which
often means Eval Controls
- scrollUpKey
- default: "\u001E" (ctrl-up). invokes :viewScrollUpScript on
current keyView
- scrollDownKey
- default: "\u001F" (ctrl-down). invokes :viewScrollDownScript
on current keyView
- keyboardMap
- similar to translationTable for keyboard, e.g.,
{keyboardMap: ["'backspace'*", "\u0008",
"'left cursor'*", "\u001C", "'right cursor'*", "\u001D", ],...}
For illustration purposes (since this is redundant), assume you want to
set ESC as the clearKey. The unicode value for ESC is 001B. So,
to turn off debug printing and override this one key (the others are still
defaulted):
Sloup
{debug: nil, clearKey: $\u001B,}
Since these options are sticky across subsequent connects (until the next
time you specify global settings), there are several strategies for multiple
files:
- set them once in a separate file that applies to all subsequent files
- make changes at beginning, reset defaults at end of file
- set default {} at beginning of each file
Here is an example of using a line continuation in entrySpec, setting
several parameters, and resetting at end:
Sloup
{fieldDelimiter: ",", stripQuotes: true,\
translationTable: ["N~", "\u00D1"], totalRecords: 3}
mysoup![]
{a: "string", b: "string"}
field1,"field2 with extra quotes"
"xxx",eN~ye
3rd,entry
BYE!
Sloup
{} // restore defaults (optional)
In the default situation, when a soup name does not exist, Sloup ignores
it and subsequent lines until it finds a valid soup name.You can also create
a new soup by appending ! to the soup name and including an array of soup
indexes (this will also find an existing soup; to add new indexes, it is
best to remove and reinitialize the soup). Before doing this, you should
feel comfortable with transferring entries into existing soups and
customizing an entrySpec. I would also recommend obtaining a soup utility so
that you can inspect entries, remove entries and remove soups (if
necessary).
To create a soup, follow the name with a ! followed by an array of index
specifications, e.g., [] is none. The soup is created as a "union" soup; on
1.x, this is created on each store, i.e., internal memory and card, leading
to some empty soups; on 2.x, the soup is created only on the current default
store.
TestSoup![]
{a: "int", b: "string"}
0 hello
1 there
BYE!
Here is an example of a different soupname line if you had wanted to
index on the integer field a:
TestSoup![{structure: 'slot, path: 'a, type: 'int},]
Indexes can be useful to applications for random access or for accessing
soup entries in a particular sort order. For examples of other index specs
see the NTK docs, or the bitmap and sound examples (next); more docs to
follow...
You can add the indexes for a soup at the end (BYE!).
This is the most complicated part of Sloup. You need to specify the
correct slot names and value types. For standard soups, you can discover
this from NTK or additional Sloup documentation or examples. In general, you
can use a soup utility (like StewPot) to inspect a soup, or print frames
with the NTK Inspector or NewtDevEnv. Or, you can DUMP! a soup using an
empty frame spec (note: this corresponds to the first soup entry, and
requires Newt for complex entries).
Here is a summary of value types that you may encounter or wish to use.
These would generally be specified as strings, i.e., enclosed in double
quotes, e.g., "int".
- string
- text
- int
- integer
- real
- a floating point number. (this should do the correct interpretation of
"decimal point" depending on locale setting, e.g., "," in Europe)
- boolean
- true or nil (false)
- symbol
- a NewtonScript symbol. When used as a value in an entrySpec, prefix with
a single quote, e.g., {labels: 'Business}. In order to include characters
other than alphanumerics and underscore _, surround the symbol with vertical
bars, e.g., {labels: '|a funny $ymbol|}
- stringWithClass
- this is basically the same as symbol, but it is used to signify a type
that will be applied later to the entire data object (used only with Names?)
- array
- comma-delimited elements, surrounded by square brackets. e.g., ["int",
"int", "int"]. If the last element in an entrySpec is an array, it can be
variable length, i.e., the last type in the array will be applied to any
"extra" data fields
- frame
- slot: value pair, delimited by commas, surrounded by curly braces, e.g.,
{a: "int", b: "string"}
- para
- a special text type, primarily intended for Names and Calendar notes.
For example, if you'd like to add a text note to a Names entry (the text
field you see when you Show:Card&Notes), include notes: ["para"] in the
frame spec. In the data entries, the text in this position up to the next
tab (or end of line if it's last) will be included as a single paragraph in
the note field
- class:string
- this indicates the type is prefixed in the actual value. For example,
"homePhone:555 1234" (the type and colon are removed before the value is
stored).
- dateTime
- on input, parses date&time string using Newton StringToDate
function, using current locale; on output, formats date&time using
DateNTime and current locale, e.g., mm/dd/yyyy hh:mm. Internally, dateTime
is represented as an integer: number of minutes since midnight Jan 1, 1904,
so you could use type int. Note: information about
seconds is lost.
- date
- on input, parses date string using StringToDate; on output, formats data
using ShortDateStr (mm/dd/yy)
- dateTimeSpecs:
- follow dateTimeSpecs: with an array of 3 format numbers:
[longDateSpec,shortDateSpec,timeSpec], which are used respectively with
LongDateStr, ShortDateStr, TimeStr, using the current locale; include nil to
omit an element. The array expression is evaluated on the Newton, so can
refer to built-in formats. For example, ROM_dateTimeStrSpecs (@66) contains
some pre-defined formats; the following table shows which formats (indicated
by "L", "S", and/or "T") are applicable to which functions(positions):
| slot | example
|
|---|
| longDateStrSpec(L) | Wednesday, July 22, 1992
|
| abbrDateStrSpec(L) | Wed, Jul 22, 1992
|
| yearMonthDayStrSpec(L) | July 22, 1992
|
| yearMonthStrSpec(L) | July 1992
|
| dayStrSpec(L) | Wed, Jul 22
|
| monthDayStrSpec(L) | July 22
|
| numericDateStrSpec(LS) | 7/22/92
|
| numericMDStrSpec(S) | 7/22
|
| numericYearStrSpec(LS) | 1992
|
| longMonthStrSpec(L) | July
|
| abbrMonthStrSpec(L) | Jul
|
| numericDayStrSpec(LS) | 22
|
| longDayOfWeekStrSpec(L) | Wednesday
|
| abbrDayOfWeekStrSpec(L) | Wed
|
| longTimeStrSpec(T) | 10:40:59 AM
|
| shortTimeStrSpec(T) | 10:40 AM
|
| shortestTimeStrSpec(T) | 10:40
|
| hourStrSpec(T) | 10
|
| minuteStrSpec(T) | 40
|
| secondStrSpec(T) | 59
|
So, you could include a date (either long or short) and/or a time.
{..._modTime: "dateTimeSpecs:[@66.longDateStrSpec, nil, @66.longTimeStrSpec]",...}
You could also combine constants (e.g., adding different month + year
components). Full details on format constants may be found in the Newton
Toolkit Documentation:
- bits, bitmap
- see Bitmaps
- picture
- see PICT
- sound, samples
- see Sounds
- resource
- see Resources. e.g., IR resource code
- binary:
- follow binary: with a specific binary class, e.g., binary:pixels