Standard Commmunication Layer Between Socket/JSChannel/Etc

[formula1]

As it is, Communication between Frames is done through JsChannel, Server and Client Websockets and I'm guessing you'll probably want to implement RTC at some point as well (not because its necessary but because its fun). However, the communication between everything is different. Theres a few things that I've thought about

The Question

What is the best way for our system to communicate with itself?

Why is this important

People (including me) generally don't want to learn 5+ different ways of doing very similar tasks

Necessities

• Events - It is native in server and has a dedicated framework for client. However, Natively it doesn't support Regular Expressions. In addition, some neither of these are asyncronous however that doesn't support Regular Expressions. However, regardless its important to note that this expects no return values
• Requests - One Requests, One (or More) Provides. Whether its Ajax, jQuery Ajax, HTTP servers, Express Server or Our Socket FrameWork. The concept remains the same
• Streams - Lower Level but an important Node "Standard". This is Important for Video/Sound Data (WebRTC), Large (possibly endless) File Data and other large/possible endless Binary Data that for whatever reason needs to be processed on the client.

Options (Can mix and match as we please)

• Callbacks - The Unfortunate Node Standard. Fortunately, there are libraries that make it a bit cleaner. In addition, making things object oriented allows for all logic to be defined in a single package (possibly reusable by making it abstract)
• Promises - This will be native in ES6 however its implemented in a much clean way in jQuery, less clean way in Mongoose and frameworks like BlueBird are dedicated to maximize its usability.
• Pub Sub - This is akin to an event emitter, express routing system and etc. Its also the way our methods and silk or anything currently works. The basic idea is that there is a "public domain" logic wants to provide for or listen to. Its not anything special or crazy but it is important to consider how its handled and if we can make one Abstract Pub Sub for everything to extend.
• Domains - This is going to sound like a terrible idea (And trust me, I know). But what you can actually do is create a Custom Error Object (Call it Return/Resolve/Done) and you can run any number of functions inside of that domain however the functions can throw a real error or return a resolved value. This can be useful for bad programming. Its not a good option but it is an option.

Our System Now

• Methods is the standard communication between User/Applications and server. It is a layer on top of WebSockets
• Silk is the standard communication between User/Applications and Applications. It is a Layer Built on top of jsChannel
• We have event support for both Methods and in small part Silk (Commands with no expectation of return values)
• We have request support for Methods and can build it on top of Silk (Commands with expectation of return)
• We have no Stream support
• there is a single queue for events to figure out what goes where

Thoughts

• jQuery is the standard of Clientside Web Development. Whether its the sexy use of css selectors, the animation api or just making ajax requests it is the bees knees compared to native. I question if we can implement something similar such as

  $("Application.editor[accepts=text/html]")
  .openFile("path/to/the.file")
  .done(function(){
  $(this).focus()
  }).error(function(){
  $("Application.packagemanager[serves=apps]")
  .focus()
  .search("Applications.editor[accepts=text/html]")
  });

It doesn't have to be that specific syntax but I believe there is potential. I believe you were starting this with silk but am not 100% sure.

• "Gulps" vs "Sips" - For the most part our communication is done in gulps and for good reason. Nobody cares to read each part of a file and compile it once "close" is called. They and I want to get my information as I expect it. However, sending>sending>sending>reading>grouping information is faster than sending>reading>grouping>sending>reading>grouping>sending>reading>grouping
• Remote Applications - Since we are using iFrames, it is possible we can load apps that aren't stored locally. There could be benefits to that such as; proprietary closed sourced frameworks, dedicated locations for doing process intensive work (https://cloudconvert.org/), Massively Multiplayer Frameworks.
• Starting and Stopping Services - One thing I'm currenlty in the process of fixing is stopping an application on the server without stopping the entire system. I'm going to be using child_process for this. However, This will also add a communication layer between the server and an application.

[zodern]

There are 2 places that need to communicate:

• Server and Client
• Silk Client and Apps Client

Eventually we might want to add a server api, but so far there hasn't been a need.

I'm guessing you'll probably want to implement RTC at some point as well

I haven't thought of it yet.

Remote Applications

I plan to have an app for users to create custom apps. The user can put in a name, url, icon and maybe a couple other settings. There is a lot of potential like what you mentioned.

Options

I prefer the promises and pub sub. Callbacks seem to get more complex.

Should streams be enabled by something like methods.stream()? The examples I have found have a node client. Do web pages treat streams as a normal response?

$("Application.editor[accepts=text/html]")
.openFile("path/to/the.file")
.done(function(){
$(this).focus()
}).error(function(){
$("Application.packagemanager[serves=apps]")
.focus()
.search("Applications.editor[accepts=text/html]")
});

I really like that idea. One concern is would it require storing the data in the DOM?

I have decided to read Javascript Patterns to better understand Javascript.

[zodern]

Starting and Stopping Services - One thing I'm currenlty in the process of fixing is stopping an application on the server without stopping the entire system. I'm going to be using child_process for this. However, This will also add a communication layer between the server and an application.

That will be important for Silk to be lightweight - starting and stoping apps when needed instead of always having them running.

[formula1]

The Common API

So, heres what we can do.

• Everything will be pub sub, regardless of who initiated it.
• Requests and Providers - Currently inorder to provide we have the add method. For requests we're using call. We can use the same thing, however I think get and give or something similar may be better. get is used in multiple libraries as well as being a common HTTP concept. give is generally library dependent, express has use/get/post, I'm guessing meteor has add.
• Events and Listeners - I've been emitting events as if its a multi user environment. I'll stop doing that. Its a bit difficult for me to stop trying but it will probably make things a lot easier. for events, we need an on and off for listeners as well as an emit for emitters. I want to make it regexified however I believe that may be a lost cause for the time being as the event syntax may change. We can also extend event emitters as we please instead of wrapping them.
• Readers and Writers - The stream API would in part be a request alter. We could put something in place for the time being with websockets or even raw get/post requests. However, this isn't probably something that is too important for the client at this point in time.

RTC

This is one of the things I'm going to do next because I think its interesting.

Server Connections API

This will be important at somepoint, however I also think it ties in strongly with security and ensuring apps don't conflict when it comes to port numbers. This may also be a new issue as well

Selector

I'm Opening up a new Issue for this