Repository navigation
Conversation
| fmt.Println("deleteme 2") | ||
|
|
||
| // auto renew reservation | ||
| err = rsv.Open(ctx, localUdpAddr, func(r *colapi.Reservation, err error) { |
There was a problem hiding this comment.
Can you explain why you chose to make this dial a connection? Intuitively I would have expected that colapi.NewReservation returns a colibri path type that implements the snet.Path interface. As such, the path could then simply be set as the path for outgoing packets on a snet.Conn, interchanginbly with other path types that all implement the shared snet.Path interface.
By opening a connection it becomes impossible to upgrade an existing snet.Conn (dialed client connection or bound listener connection) from a scion path to a colibri path.
There was a problem hiding this comment.
The way the API is design forces the user to not make mistakes: since every renewal changes the path, the path itself is contained and managed inside the object, private to the user.
I can add a call to retrieve a copy of the path, but how would you use it? You mention you'd like to upgrade an existing snet.Conn. Can you give me a use case for this? (maybe I have overseen something in my scenarios). There is (will be) a special case for QUIC, in where I must be able to construct a quic socket with a reservation, but this is left out for later.
Regarding Listen: I don't even think it makes sense to Read from a reservation, but I kept the call; I should probably remove it. A call to Listen would receive the (not yet implemented) call from sciond to authorize the reservation or not. But once the reservation is set up, it will simply receive packets on that socket (IP-port), and any regular call to Listen will work (e.g. the server in hellocolibri). Do you think they should fail to accept packets from a reservation?
There was a problem hiding this comment.
The way the API is design forces the user to not make mistakes: since every renewal changes the path, the path itself is contained and managed inside the object, private to the user.
That exact behavior could be encapsulated inside a snet.Path though as I see it? The "actual" path (the bytes to use in packet headers) is returned from the Path() getter on snet.Path and can hence be updated whenever the reservation is renewed.
I can add a call to retrieve a copy of the path, but how would you use it? You mention you'd like to upgrade an existing
snet.Conn. Can you give me a use case for this? (maybe I have overseen something in my scenarios). There is (will be) a special case for QUIC, in where I must be able to construct a quic socket with a reservation, but this is left out for later.
As for use-cases: Generally Colibri only makes much sense for connection-oriented traffic, right? So stuff like QUIC and RTP/WebRTC. Here are two use cases using either protocol:
- A file download using QUIC: A client dials into a server and initiates a download. In this case the client is not interested in having guaranteed bandwidth to the server, as all the client needs to do is send the initial request (e.g. HTTP GET) and send QUIC ACKs. The server sending the file to the client, however, might want to send using Colibri. In this case the server (=listener) accepts a new client connection and would then have to upgrade the connection to the client to Colibri after the connection was already established.
- A WebRTC call as in my thesis. Both peers in a call are basically equal and there is no big client-server distinction one a call is running. However, the underlying call setup may involve one peer using a listener and the other peer dialing into that listener, or using STUN to patch together two dialed connections. In either case, it should be possible for either a dialed connection or a bound listening connection to upgrade to Colibri. As in the previous example, a bound listening connection can only be upgraded to Colibri after the connection is already established. In case of using STUN, a dialed connection also needs to have the ability to switch to Colibri after initially not using Colibri, since STUN involves first connecting to a STUN server, then re-using the same connection to continue talking to the call peer (so it actually involves a change of the destination).
I would imagine using path processors to choose Colibri paths:
- There might be a case where a switch away from Colibri back to regular SCION is wanted. E.g. the only available Colibri path has huge latency because it takes a detour, and causes bad QoS/QoE. Switching to a more direct SCION path would provide lower latency.
- Or simply a switch from one Colibri path to another (e.g. initial choice was bad), all without having to close and create new connections.
Also creating a reservation involves another request which would increase the time to connectivity. Therefore I think it would make a lot of sense to start many Colibri connections as regular SCION connections and parallelize the initial packet sending over SCION with the requesting of an e2e reservation. Once the e2e reservation is done and a Colibri path is available, the connection could be upgraded to use the Colibri path.
Regarding Listen: I don't even think it makes sense to Read from a reservation, but I kept the call; I should probably remove it.
Why not Read()? Open simply dials a connection and it needs to be possible to read and write packets just like on a regular scion connection. Or what do you mean?
A call to
Listenwould receive the (not yet implemented) call fromsciondto authorize the reservation or not. But once the reservation is set up, it will simply receive packets on that socket (IP-port), and any regular call toListenwill work (e.g. the server inhellocolibri). Do you think they should fail to accept packets from a reservation?
To make sure I understand correctly: If a listener accepts an e2e reservation, then is closed, and another listener is re-opened on the same port, I guess inherently by the design of Colibri the admission provided by the previous listener on the same port stays valid? Only renewals would then be handled by the new admission callback used by the new listener.
It might make sense to go for the more cautious approach. Who knows what kind of shenanigans could otherwise become possible. Each listener can keep a record over admitted Colibri clients – if it receives an incoming packet that uses a Colibri path and the client is not listed as having been admitted via the listener's admission callback, the packet could be dropped/rejected. This would happen when a now closed listener on the same port previously accepted the reservation. The client would have to first request a new e2e reservation that needs to be accepted by the new listener on the same port.
There was a problem hiding this comment.
The way the API is design forces the user to not make mistakes: since every renewal changes the path, the path itself is contained and managed inside the object, private to the user.
I can add a call to retrieve a copy of the path, but how would you use it? You mention you'd like to upgrade an existing snet.Conn. Can you give me a use case for this? (maybe I have overseen something in my scenarios). There is (will be) a special case for QUIC, in where I must be able to construct a quic socket with a reservation, but this is left out for later.
The same problem exists for SCION paths that also expire (not quite so quickly, but the issue is identical). This is not addressed in snet etc. but is left to the applications. Imho, a high level API as proposed here should not be specific to colibri, but should be more generic and integrate the logic for renewal, fallbacks etc in one package. I agree with Jonas that enabling/disabling the use of Colibri for an existing connection seems like an important feature, not least because of the potential need to fallback from Colibri to best-effort SCION.
My suggestion would be to add colibri support to "pan" (see #187, which is still incomplete and has been collecting dust, unfortunately) instead of building a separate high-level API for Colibri only.
There was a problem hiding this comment.
That exact behavior could be encapsulated inside a
snet.Paththough as I see it? The "actual" path (the bytes to use in packet headers) is returned from thePath()getter onsnet.Pathand can hence be updated whenever the reservation is renewed.
I just made the Reservation implement snet.Path: it can now be used directly if so wanted.
As for use-cases: Generally Colibri only makes much sense for connection-oriented traffic, right?
No no, it makes sense also for connectionless protocols.
So stuff like QUIC and RTP/WebRTC. Here are two use cases using either protocol:
- A file download using QUIC: A client dials into a server and initiates a download. In this case the client is not interested in having guaranteed bandwidth to the server, as all the client needs to do is send the initial request (e.g. HTTP GET) and send QUIC ACKs. The server sending the file to the client, however, might want to send using Colibri. In this case the server (=listener) accepts a new client connection and would then have to upgrade the connection to the client to Colibri after the connection was already established.
Keep in mind that in this scenario, the server will setup a new reservation, meaning that the client must be able to accept it, either via whitelists/blacklists or via a callback to grant the setup. The same applies for renewals.
But I understand the scenario. With the current API, the server gets the destination from the incoming connection, creates a reservation, and opens it. It then proceeds to write packets to the Reservation.
- A WebRTC call as in my thesis. Both peers in a call are basically equal and there is no big client-server distinction one a call is running. However, the underlying call setup may involve one peer using a listener and the other peer dialing into that listener, or using STUN to patch together two dialed connections. In either case, it should be possible for either a dialed connection or a bound listening connection to upgrade to Colibri. As in the previous example, a bound listening connection can only be upgraded to Colibri after the connection is already established. In case of using STUN, a dialed connection also needs to have the ability to switch to Colibri after initially not using Colibri, since STUN involves first connecting to a STUN server, then re-using the same connection to continue talking to the call peer (so it actually involves a change of the destination).
I am unfamiliar with STUN, but I believe that this scenario is accepted by the current API: client A dials to a STUN server, which in turn dials to a peer B. After that, client A opens a reservation to the destination, which must be the STUN server, and starts using that Reservation.
In general, the action flow for upgrading a connection to colibri would look like this:
- client has an existing, non colibri, connection to another peer that we will call server
- client sends data to the server using the non colibri connection
- client creates a new reservation to server, still can send data to server via the non colibri connection
- client opens the colibri reservation
- client can now send data using the colibri
Reservationor the non colibri connection, or both.
Only caveat here is the dispatcher: we cannot register to listening sockets using the same port, as they would clash.
This also makes even more outstanding the fact that receiving packets on a colibri connection is no different at all compared to regular scion packets, when looking from the perspective of the listener.
I would imagine using path processors to choose Colibri paths:
- There might be a case where a switch away from Colibri back to regular SCION is wanted. E.g. the only available Colibri path has huge latency because it takes a detour, and causes bad QoS/QoE. Switching to a more direct SCION path would provide lower latency.
Yes, this would be desirable. Again, the abstraction let us not stop here, as hidden paths or epic are also paths that could be selected from a path processor. All of them could be seen as snet.Path
- Or simply a switch from one Colibri path to another (e.g. initial choice was bad), all without having to close and create new connections.
With the current API + Reservation been an snet.Path you don't need to close connections.
But once again, a caveat: if closing a connection or establishing it imply changes to the internal state of the connection, we might have to do it anyways. E.g. QUIC should have no flow control other than constant bit rate when the underlying connection is colibri.
When closing and opening a connection is no operation, the result of switching back and forth between colibri and regular scion is accepted by the current API, even without the snet.Path interface.
Also creating a reservation involves another request which would increase the time to connectivity. Therefore I think it would make a lot of sense to start many Colibri connections as regular SCION connections and parallelize the initial packet sending over SCION with the requesting of an e2e reservation. Once the e2e reservation is done and a Colibri path is available, the connection could be upgraded to use the Colibri path.
Why not
Read()?Opensimply dials a connection and it needs to be possible to read and write packets just like on a regular scion connection. Or what do you mean?
Read gives the impression that this colibri connection can be used bidirectionally, which is not true. In can be used to read packets, but with a regular call to Listen and Read it would be the same. The call is in the API because removing it would prevent easily reading back e.g. ACKs and short responses, so I guess it will have to be very well documented to remove the sensation of reservation when reading. Note that for the same reasons, there is no call to Listen.
To make sure I understand correctly: If a listener accepts an e2e reservation, then is closed, and another listener is re-opened on the same port, I guess inherently by the design of Colibri the admission provided by the previous listener on the same port stays valid? Only renewals would then be handled by the new admission callback used by the new listener.
The mechanism of how to accept or deny a setup/renewal by the endhost is not decided yet. Originally it was done via white/black lists, but I would love to have a design in which we have a callback.
That been said, the listener allowing/denying connection setups would always be present, and a new "ingester" would be spawned (maybe on a different port) if the setup was accepted. E.g. via a response back from the "accepter" with a new port, or a different mechanism.
It might make sense to go for the more cautious approach. Who knows what kind of shenanigans could otherwise become possible. Each listener can keep a record over admitted Colibri clients – if it receives an incoming packet that uses a Colibri path and the client is not listed as having been admitted via the listener's admission callback, the packet could be dropped/rejected. This would happen when a now closed listener on the same port previously accepted the reservation. The client would have to first request a new e2e reservation that needs to be accepted by the new listener on the same port.
The mechanism in colibri forbids a reservation from being created or renewed if the endhost doesn't accept it. So the listener would never see colibri packets if it didn't accept the reservation.
Please remember that this exact part is still a TODO in the code, as we are discussing here how we want to communicate the endhost with the service.
There was a problem hiding this comment.
The same problem exists for SCION paths that also expire (not quite so quickly, but the issue is identical). This is not addressed in snet etc. but is left to the applications. Imho, a high level API as proposed here should not be specific to colibri, but should be more generic and integrate the logic for renewal, fallbacks etc in one package. I agree with Jonas that enabling/disabling the use of Colibri for an existing connection seems like an important feature, not least because of the potential need to fallback from Colibri to best-effort SCION.
Partially agree: unifying all paths will also expose differences: the high level actions that make sense for colibri might not make sense for others. E.g. a fallback in colibri doesn't make sense unless it can be described to the service itself. It then must be characterized somehow by a description such as minimum reservation bandwidth X iff same latency, but X-2 otherwise, etc. Or send a list of reservations to the service, instead of just one.
My suggestion would be to add colibri support to "pan" (see #187, which is still incomplete and has been collecting dust, unfortunately) instead of building a separate high-level API for Colibri only.
Trying to unify the paths under a common API would only slow down the design of the colibri API. Furthermore, since I know nothing about PAN, integrating all of it into PAN would delay things even further, at least for me.
| } | ||
|
|
||
| fmt.Println("deleteme 1") | ||
| rsv, err := colapi.NewReservation(ctx, scionNet, daemon, dstAddr, 9, 0, func(a, b libcol.FullTrip) bool { |
There was a problem hiding this comment.
When calling this twice in succession it would try to create two separate reservations. Would that work? Or should it return the same reservation for both calls?
Also might there be a use case for splitting this into two separate functions without going down to the level of the basic API? One function that simply returns a slice of libcol.FullTrip, and another function that selects one libcol.FullTrip of such a slice given one or more sorting predicates. This might be useful to query all possible libcol.FullTrips in advance before making a selection later (I assume the querying, daemon.ColibriListRsvs(), takes some time).
When the reservation fails because the requested bandwidth is not supported it would make sense to be able to automatically fall back to the second-best path chosen via the given predicates and so on, until one supports the given bandwidth. In general, the available bandwidth should ideally be a parameter to base sorting on, just like the hop count and expiration time, such that trade-offs between the three of these can be defined rather than sorting based on only expiration time and hop count and hoping the requested bandwidth is supported. But it seems like the max bandwidth cannot be known in advance and can only be discovered by requesting an e2e reservation?
There was a problem hiding this comment.
When calling this twice in succession it would try to create two separate reservations. Would that work? Or should it return the same reservation for both calls?
It would create two distinct reservations. It should work 😃 Note than creating a reservation is different from having an approved reservation, as the later happens when calling Open.
Also might there be a use case for splitting this into two separate functions without going down to the level of the basic API? One function that simply returns a slice of
libcol.FullTrip, and another function that selects onelibcol.FullTripof such a slice given one or more sorting predicates. This might be useful to query all possiblelibcol.FullTrips in advance before making a selection later (I assume the querying,daemon.ColibriListRsvs(), takes some time).
Current API has it split in two parts: New and Open. The results of both are not visible to the caller, though. It might make sense to let the user "see" the result of the combination, as this is abstracted away from them through the sorting functions. Note that at the moment the API user can have the first sorting function always returning false just to keep track of the seen FullTrip in e.g. a map. Although this feels like a hack instead of a valid use of the API.
There was a problem hiding this comment.
When the reservation fails because the requested bandwidth is not supported it would make sense to be able to automatically fall back to the second-best path chosen via the given predicates and so on, until one supports the given bandwidth.
I would prevent this: when the reservation fails, the user can react quickly enough to emit a new reservation request, that will not take longer to be effective than using a "fallback" mechanism.
The scenario where the renewal fails is more time critical, to my eyes: when a renewal fails, the user has very limited time to get another reservation (less than 8 seconds), so an arbitrary number of failures cannot be solved that way, while it would benefit from having an in-protocol fallback mechanism (which we don't have), and this way the reservation renewal will be resolved in one go.
In general, the available bandwidth should ideally be a parameter to base sorting on, just like the hop count and expiration time, such that trade-offs between the three of these can be defined rather than sorting based on only expiration time and hop count and hoping the requested bandwidth is supported. But it seems like the max bandwidth cannot be known in advance and can only be discovered by requesting an e2e reservation?
You are right, and I forgot to include the allocated bandwidth in the segment reservation look, i.e. ReservationLooks should contain much more information, like min/max/alloc bandwidth.
Note that the sorting functions do not have any restriction in what they use to sort, or if they want to keep state, etc. Any function has access to the FullTrips they are sorting, which, as mentioned right above, will contain information such as allocated bandwidth.
There was a problem hiding this comment.
I would prevent this: when the reservation fails, the user can react quickly enough to emit a new reservation request, that will not take longer to be effective than using a "fallback" mechanism.
Yes, in the end this boils down to how easy to use the API should be, how many standard use-cases should already be covered by built-in functions, and how much a user of these APIs could do wrong if they are trying to do something that isn't already done automatically. This no-fallback method could be provided in addition to a method implementing automatic fallback.
You are right, and I forgot to include the allocated bandwidth in the segment reservation look, i.e.
ReservationLooksshould contain much more information, like min/max/alloc bandwidth.
Ok if the max and min bandwidth can already be known ahead of time, and it is 100% accurate (probably not), it would resolve the above discussion about performing a fallback when the requested BW is not allowed.
There was a problem hiding this comment.
Yes, in the end this boils down to how easy to use the API should be, how many standard use-cases should already be covered by built-in functions, and how much a user of these APIs could do wrong if they are trying to do something that isn't already done automatically. This no-fallback method could be provided in addition to a method implementing automatic fallback.
Not only that: the failure may alter the way that the reservations were originally sorted (e.g. if no bandwidth is avail in AS X, avoid it). This means that the fallback mechanism should be dynamic (evaluated at that point in time, with the error information), which at the moment is not possible unless running a function.
Ok if the max and min bandwidth can already be known ahead of time, and it is 100% accurate (probably not), it would resolve the above discussion about performing a fallback when the requested BW is not allowed.
min, max, and alloc bandwidth are parameters of the segment reservation. They are accurate for them, but they do not represent the current load of the segments.
I do not agree with any statement that would summarize in "a mechanism in which the client falls back to the next best reservation is good and necessary"; let me explain why, and try to prove it:
- any fallback mechanism in the client is no more efficient than the evaluation of a function dynamically if a reservation setup or renewal fails. If a fallback mechanism could resolve a reservation on a second or even a third try, so would do a function, just by returning this preferences everytime is called.
- Any such preferences expressed in a list that would be present in a fallback mechanism can be represented by the repeated invocation of a function. Thus the fallback mechanism is not necessary.
- Because the last two statements, any behavior of the fallback mechanism is a subset of the error function mechanism.
- Furthermore, the fallback mechanism cannot express dynamic behavior, such as acting on failure of a specific AS (e.g. avoid AS X when possible). This is a very common scenario, and any "good" mechanism should cover it. The fallback mechanism is then not "good".
- As a corollary to the above statement: the fallback mechanism doesn't cover all the behavior that the error function does, and is thus incomplete in the possible behavior cases.
| scionNet := snet.NewNetwork(localIA, dispatcher, sciond.RevHandler{Connector: daemon}) | ||
|
|
||
| fmt.Printf("server at %s\n", udpAddr) | ||
| conn, err := scionNet.Listen(context.Background(), "udp", udpAddr, addr.SvcNone) |
There was a problem hiding this comment.
The listener just always auto acks reservations? Or is there a method to handle and respond to reservation requests?
There was a problem hiding this comment.
That's still a TODO in the code (and the design). The initial idea is this:
- The listener tells
sciondor directlySvcCOLit accepts connections. It can blacklist sources as well. - This
listenmessage stating it accepts reservations expires after e.g. 60 seconds, and has to be renewed before it expires. - When a reservation setup/renewal arrives to
SvcCOL, it consults the table of destination endhosts and checks for blacklists andlistenmessages. It fails/accepts the reservation appropriately.
What I would really like to have in the API is a callback for the listener, that runs every time a reservation is received for setup or renewal in the destination endhost. But I cannot find a way to do so elegantly 😞 🐼 We should have a meeting to talk about this, and update the PR afterwards.
There was a problem hiding this comment.
What you mentioned above, passing a callback handler into Listen() for Colibri sounds reasonable to me. Or alternatively having a member/function on the type returned from Listen() that configures the reservation request handler.
This callback handler would be registered with sciond/colibri service for the specific port that the listener is listening on. If the listener is closed, the callback is unregistered. If a reservation request comes in to sciond/colibri service for a specific port, the callback registered for that port is called and the result (accept or deny) is used to reply to the reservation request. If no callback is registered the request is automatically denied. This sounds like what you were thinking of. Can you point out the problems of this approach?
juagargi
left a comment
juagargi
left a comment
There was a problem hiding this comment.
Reviewable status: 0 of 5 files reviewed, 3 unresolved discussions / 0 of 1 LGTMs obtained / 0 of 1 approvals obtained
_examples/hellocolibri/hellocolibri.go, line 70 at r1 (raw file):
Previously, JonasGessner wrote…
What you mentioned above, passing a callback handler into
Listen()for Colibri sounds reasonable to me. Or alternatively having a member/function on the type returned fromListen()that configures the reservation request handler.This callback handler would be registered with sciond/colibri service for the specific port that the listener is listening on. If the listener is closed, the callback is unregistered. If a reservation request comes in to sciond/colibri service for a specific port, the callback registered for that port is called and the result (accept or deny) is used to reply to the reservation request. If no callback is registered the request is automatically denied. This sounds like what you were thinking of. Can you point out the problems of this approach?
After our offline (from this PR) discussion we have decided to go for the following:
- There will still be a call from
sciondto SvcCOL setting black/white lists. This is necessary to avoid being DDoS by a big number of acceptance requests, but as mentioned, it's not very fine grained. - There will be a callback registration system, so that the application will run a function in its memory space, returning an answer yes/no accepting the setup/renewal. The function won't be called if the first step, with the white/black lists, determines the setup/renewal won't be approved.
The details of this design are discussed somewhere else (at the moment in TeX form, but will be migrated soon).
Also show how to use an existing conn to send packets with colibri paths.
In particular, the server now requires to periodically indicate it can receive reservations (white/black lists).
Requests and responses now have validator fields. Added calls to create the authenticators for the requests. Added calls to validate responses.
DRKey is no longer optional, the local topologies include configuration for DRKey. The local topologies also include the delegation entries for the scion daemon for the two protocols "colibri" and "piskes". Updated the local address of the scion daemons for 111 and 112. Added a timeout to connect to the scion daemon.
|
Obsolete. |
3 participants
Introduces a COLIBRI client example.
Pending:
go.modThis change is