Updated as of version 3.3a8.
This implementation started from Java source on the main VNC site . I ported it to Waba -- but with a subset of virtual machine (JVM) and built-in libraries that runs on many platforms (Palm, WinCE, Newton,... or any platform that supported a regular JVM). This Waba client ran on Windows, but it was unsuccessful running in Waba for the Newton; !!! further Waba work needed to address current communication (Socket) problems, and future Graphics issues too. Of course, I could have just ported the C++ version, but that's not my favorite language, plus the Newton C++ tools and constraints would make my life less productive, even if the resulting Waba/NewtonScript versions weren't as speedy. To partially address performance issues, Paul Guyot has provided several C++ libraries: VNCNative for several optimized graphic methods to fill rectangle and convert bitmaps, and Zlib to support Zlib encoding: decompression for Viewer (and maybe compression for Server).
Still, the port to Waba was a useful exercise, since this resulted in an unthreaded, state machine model. I then ported the mostly-working Waba version to NewtonScript (NS), figuring that it would be easier to debug, and if the NewtonScript version was too slow, the Waba version would be slow too (or slower); if NS version was acceptable, we'd have a decent working model/benchmark for improving the Waba version.
An initial porting problem was that NewtonScript supports only 30-bit integers, rather than 32-bit as in Java/Waba or C++. So, I had to rewrite the DesCipher class (used for initial password encryption/authentication) to use my own Band, Bor, Bxor, BShift functions to deal with 32-bit data inside a binary object (4 8-bit bytes) rather than using NS integer. Messy but workable.
The current version of newtVNC is NOS 2.1-only; although there were earlier versions that could run on NOS 2.0, there were a number of limitations (and no users that I was aware of):
I will make source available eventually; of course, if someone has a specific idea they'd like to pursue, I could provide sooner. I'd like to start cleaning up/documenting code.
The Newton does not support color -- only gray up to 16-levels. VNC servers provide a minimum of 8-bits/pixel (0-255) of color information, which need to be mapped to the Newton's RGB values, which then be used directly (e.g., fillPattern) to display as gray, or mapped to a 4-bit gray value (0-15) for 16 grays, 2-bit gray value (0-3) for 4 grays, or 1-bit b&w value (0-4) for black, white and 3 dithered grays; when update rectangles aren't too small, b&w is surprisingly legible.
For newtVNC server, since only 256 colors are used, the color representation is not ideal/accurate; if backlight is turned on, a greener set of colors is used (similar to NPDS); if anyone would like to try fine tuning these colors in NTK Inspector, I can tell you how to compute/patch the two color arrays.
Most VNC servers/viewers do not currently/widely support actual "color maps", so it has been necessary to encode colors as "true" rgb bit values; for 8-bit pixels, this is bbgggrrr, which isn't really adequate; perhaps if newtVNC used 16-bit pixels (which would double the amount of data sent), it could use 5-bits for each r,g,b value -- this option may be added in a future update, and/or I will look into using color maps if enough viewers indeed support them
Most VNC versions assume a synchronous communication stream. A NS application can't really wait for input -- it has to set things up asychronously via callbacks. "RFB" (remote frame buffer) protocol has different length responses depending on the type of data, so the Newton has to keep changing how much data to expect. In some cases, iterative loops had to be implemented via callbacks also; initially, it read a few bytes of data for each individual rectangle, but obviously this had a lot of communication overhead; another version requested as many rectangles as available all at once -- this minimized communication but led to long delays and memory issues (where it would have to allocate a large, possibly ~128K byte buffer as a VBO); the current approach balances these tradeoffs so that it tries to transfer up to ~5K at a time; this also allows for periodic updating of progress while minimizing heap and avoiding VBOs. Amazingly, the Newton keeps up pretty well.
newtVNC server also implements a state machine for output, so that it can send screen updates in smaller pieces.
newtVNC server basically uses Raw (regardless of remote viewer preference), though it will use RRE or CoRRE (if supported by viewer) for same color-bitmaps and to pre-erase larger bitmap update areas. Compression is typically more processo and memory-intensive than decompression, so trying to provide Zlib was not successful on MP2K, and on MP2100, initial tests did not indicate any noticeable speed or size advantage (Paul will investigate). Initial Hextile support was slower than Raw with RRE/CoRRE same-color optimization (perhaps due to overhead of outputting individual hextile records), so it was not included.
newtVNC viewer supports Raw, RRE, CoRRE, Hextile and Zlib. Initial experiments with ZlibHex and Tight encodings were not faster, despite smaller number of bytes transferred than even Zlib, probably due to extra decoding overhead. Zlib is generally fastest, followed by Raw (fast primarily due to its use of Paul's native drawBitmap method), and then Hextile (small, but decoding overhead). Of course, speed of remote processor and network, and further optimizations on Newton could change the timings and rankings. Your mileage may vary.
In initial versions (3.3a1,a2), newtVNC viewer drew directly (and temporarily) to the screen; this had the virtue of being easy to implement and could demonstrate whether this would be useful at all; the major drawbacks included refresh problems with popup views -- getting overwritten by VNC or leaving holes in the display area, inaccessibility of character recognition (since it caused the screen to clear) and no support for the CopyRect encoding. 3.3a3 started maintaining a local bitmap which addressed all of these problems; for a 320x400 screen, a Newton 16-gray image would require a ~56K bitmap (2 pixels/byte); however, this is stored in a default-store-based VBO (virtual binary object); it seems prohibitive to maintain 16-gray bitmap for entire remote screen (e.g., 1024x768 / 2 ≈ 394K), though scrolling would be instantaneous once the entire initial refresh were done, and it would be possible to maintain a scaled-down "map" of remote screen for navigational purposes (Overview ≥ 3.3a6); incremental updates would take longer if non-visible parts of screen are also changing.
Paul Guyot provided several C++ routines for fillRect and drawBitmap, which handle all gray levels, so the rest of this paragraph is no longer relevant, but it's retained for historical interest. The server sends Raw bitmaps for icons or detailed areas in any encoding, or for entire screen if Encoding: Raw is specified. There are two internal methods for implementing this; these labels are somewhat misleading here since both are Reliable, and the Reliable method might be Fast(er) for certain data:
In 3.3a2, there was a Draw Raw Pixel preference for Reliable vs. Fast. However, currently VNC automatically selects the bitmap rendering method automatically based on the resolution.
CopyRect. Once VNC (3.3a3) started maintaining a local bitmap, it was possible to support CopyRect. There are two implementation paths: if the source and destination rectangles don't intersect, it uses the :ViewIntoBitmap method to copy the rectangle all at once within the same bitmap; otherwise, it copies the source bitmap to the destination (itself) in strips via a temporary buffer (to avoid memory issues), either top-down or bottom-up to avoid overwriting portions not yet copied.
When newtVNC server detects a new open application (or global view like a popup) -- by mapping through GetRoot():childViewFrames(), newtVNC updates the app's area, including border, and patches several methods in the app (and its child views) such as viewQuitscript, and either viewDrawScript or a class-specific change script so that it can be notified about incremental changes to the screen. Ideally, each patched viewDrawScript would notice the minimal changes necessary via :GetDrawBox, and these would be sent to newtVNC. However, for many objects the refresh does not propagate to the parent (app) and it is not done via viewDrawScript even locally.
For example, a clMonthView might notify of changes through monthChangedScript or Dirty, or at a higher level through newTime (in protoDatePicker); closing a view might not always trigger a change elsewhere, so a viewQuitScript was needed (??? maybe this isn't necessary since more objects notify about changes now). The patching occurs for an app once, so does not effect children that might be opened later or created dynamically. BuildContext is a possibility, though it would only notice new objects. This patching/notification mechanism still needs much tuning to be more complete, while minimizing the number of update requests and size of update areas.
The actual update is captured via GetRoot():ViewIntoBitmap. One issue is that this usually(?) triggers a viewDrawScript, so a status flag is needed to block this from generating yet another update. ViewIntoBitmap does not capture certain screen additions/decorations, such as text insertion caret and highlightinng. GetCaretBox would provide coordinates of caret (if any); after checking that it's contained in current update rectangle (possibly split due to bitmap slicing), it could be or-ed into bitmap before it is sent to remote viewer (??? caret bitmap itself isn't documented?). Hilighting would be trickier since AFAIK, there is no nice function to obtain x,y coordinates of the current hilite area, or reverse mapping from a range of characters to x,y. (wanted: an inverse of :PointToCharOffset; I did something like this using StrFontWidth for Newt's Cape to generate link locations for 1.x books, but it's rather complicated, and unclear whether it's worth the extra code or time overhead).
newtVNC server attempts to eliminate unnecessary updates, e.g., contained already in pending requests by basically comparing intersection of proposed update rectangle with existing pending rects. There are still situations where an app might refresh twice, or animates or updates frequently could still cause extra refreshes, e.g., the "Card" app. Once an update has been sent, it seems necessary to send any requested update again, since something may have changed even after a brief delay.
newtVNC should unpatch apps when disconnected; hopefully there are no incompatibilities with other 3rd party apps. don't remove memory card or Rotate while newtVNC server connected. P>when NIE connected on MP2K/eMate, closing a full-screen app can often be excruciatingly slow (due to Newton system bug)
Character recognition. newtVNC has a child clParagraphView underneath the screen view; it's offset above the main screen and clipped to hide caret and text. Generally, the various settings of click/gesture scripts and flags allowed the screen view to see gestures, and for the paragraph to interpret characters. When a character is recognized, it erases the entire paragraph area -- not just the top line; this also erases the main screen view too; disabling ink didn't prevent this (plus made HWR less friendly). So, HWR was disabled in 3.3a1,a2. Other solutions might include a separate Palm-like writing area. But since 3.3a3 maintains a local bitmap, handwriting can now occur directly on the main screen. Unfortunately, the "hilite" and "doubletap" gestures are no longer accessible if handwriting is on; there is a preference in 3.3a4 to turn handwriting off.
newtVNC server uses InsertTabletSample to simulate pen input. This generally works quite well for tapping, drawing, handwriting and simple gestures, but does not appear to support hiliting/dragging, e.g., hilighting text, selecting apps (in Extras icon mode), and dragging windows.
newtVNC server uses HandleKeyEvents to simulate keyboard input. This works quite well, though has not been tested extensively with multiple chorded key combinations (and some preferences may be needed to map for different systems). Also, certain key codes (like Esc and F1-F12 keys) are unknown, so not currently provided.
The Newton does not provide a prototype scrollbar object (though perhaps there's something built-in/undocumented in NewtWorks that could be borrowed). So, each scrollbar consists of two picture buttons (using built-in/rotated icons) and a slider. The Newton provides a built-in horizontal slider, but no vertical slider. To see if I had overlooked this or find out if someone else had already implemented and posted one, I did a quick newsgroup search (via "DejaNews") -- I found a reference to a DTS vertical slider example; what was amusing is that the posting was from me ~3 years ago (and I had totally forgotten about it)! I modified this to scroll from top-to-bottom, and added a true rather than dithered gray fill; ??? it still needs some minor tweaking to get the dimensions to match exactly.
newtVNC uses a localization frame. So, if someone was interested in adding labels, menu commands and status/error messages in other languages, it should be straightforward to add a new language frame.
Although having separate Viewer and Server apps might make it possible to run both simultaneously, I'm doubtful that the Newton (at least MP2K) has adequate resources for both; single newtVNC .pkg is easier to maintain due to large code overlap; and it's smaller: ~105K vs. separate: 2 * ~85K?