• 8 Posts
  • 1.14K Comments
Joined 3 years ago
cake
Cake day: July 7th, 2023

help-circle














    1. How many grep-like ops per file?
    2. Is it interactive or run by another process?
    3. Do you know which files ahead of time?
    4. Do you have any control over that file creation?
    5. Is the JSONL append only? Is the grep running while the file is modified?
    6. How large is very large? 100s of MB? Few GB? 100s of GB? Whether or not it fits in memory could change the approach.
    7. You’re using files, plural, would parallelizing at the file level (e.g. one thread per file) be enough?
    8. How many files and how often is that executed?





  • I’ve never worked on a codebase where using ORMs wasn’t better than rolling your own queries. What are people writing that they actually need the marginal performance gains? And even if that’s worth it, why not just use raw queries in your critical paths?

    Every time I have to write or modify raw SQL it feels like I’m throwing away all my static checking features and increasing the chance of bugs, because I have no idea of the query matches my schema or if it’ll blow up at runtime.