Explore
New notes
Show Notes
Replying to npub1t0nyg64
Thanks all! I wasn't quite ready to launch this thing, but I really appreciate the feedback, cheers! I was having some stability issues which was why the site was down for a while earlier today. It should be fixed now.
0
0
0
0
Replying to npub1xtscya3
Yes, I have seen nostrdb. I think it's a great idea, and that it will be a big benefit for native clients. Let me know if I can help in any way! I was thinking about it a bit and maybe flatbuffers is a bit overkill for the indexed nostr events in strfry, and I'm working on a branch that uses a custom event representation similar to nostrdb. I usually use flatbuffers by default because the generated code is convenient and it's easy to evolve the schema in a backwards compatible way. Also it comes with some nice utilities for debugging and securely parsing untrusted data (not needed here). BUT the core nostr event layout is pretty much fixed and saving some space here is important not least because it will let us pack more records into the page cache. I'll keep you posted on how that goes. Cheers!
0
0
0
0
Replying to npub1drvpzev
600ms seems too long for that set size. I will take a look at the performance when I have my Rust harness working. Let's make sure it's working the same way as the C++/JS impls first, and then we can see if there any speed-ups. One optimisation I did in the C++ was to ensure that the XORing is always done word-wise, instead of byte-for-byte. In fact, with attention to the alignment, I got the compiler to emit PXOR SIMD instructions to do multiple words in a single instruction. That's very cool about the UniFFI bindings, I'm looking forward to it!
0
0
0
0
Replying to npub1drvpzev and npub1u8lnhlw
I only have a very rough understanding of how minisketch works, but from what I know it is the opposite. Minisketch is very good with small set sizes -- so good that it approaches the information-theoretic bounds of what is possible, bandwidth-wise. However, look at the first graph on the minisketch site: Here you can see that the CPU time required is exponential in its "capacity" parameter (note the Y axis is log-scale), and will become impractical in the low thousands. The text hints at this too: "For the largest sizes currently of interest to the authors, such as a set of capacity 4096 with 1024 differences". OTOH, negentropy works predictably well into the millions or tens of millions of events. By pre-computing fingerprints as described in Aljoscha Meyer's paper, this will scale to arbitrarily large sizes. Some other downsides that I see with minisketch are that you need to choose an appropriate capacity before you begin the reconcilliation, otherwise it will fail (?). How do you choose that? Also, minisketch's performance depends on the field-size, which I believe is basically how large the IDs being reconciled can be. The largest example shown is 64 bits, which is uncomfortably small if trying to avoid collisions. negentropy by default uses 128 bits, but can scale this arbitrarily high with only linear overhead in bandwidth and CPU. By contrast, negentropy requires almost no tuning. The default parameters I have selected seem to work well for both small and large set sizes, and with small and large symmetric differences between the two sides. I am in the process of adding negentropy support to gensync, which is a general benchmarking framework for set reconcilliation protocols. I'll post any updates about my progress here: https://github.com/nislab/gensync/issues/9
0
0
0
0
I just landed a new feature in strfry: "strfry router" Docs: https://github.com/hoytech/strfry/blob/master/docs/router.md I kind of think of this like "nginx for nostr". It lets you manage many up/down/both streams to remote relays using a single process and config file, and you can edit that config file without interrupting existing streams (mostly). You can also use nostr filters and/or plugins to filter the traffic in both up and down directions, independently.
0
0
0
0
Hey all! I'm going to be doing a talk on nostr and the architecture of the strfry relay. It's next week at the CppNorth conference in Toronto, Canada: https://cppnorth.ca/speaker-doug-hoyte.html If anyone's around, it'll be a pretty fun conference! You can enter NOSTR in the checkout to get a discount on the ticket.
0
0
0
0