Med-Mastodon

2 posts2 participants1 post today

Free Software FoundationAssigning your copyright to the FSF helps defend the GPL and keep software free. Thanks to Rongzhao Yan, Paolo De Santis, and Maximilian Küffner for assigning their copyright to the FSF! More at: <a href="https://u.fsf.org/463" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://u.fsf.org/463</a> <a href="https://hostux.social/tags/Emacs" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Emacs</a> \<a href="https://hostux.social/tags/Wget" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Wget</a> <a href="https://hostux.social/tags/CopyrightAssignments" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#CopyrightAssignments</a>

Preston Maness ☭I've mirrored a relatively simple website (redsails.org; it's mostly text, some images) for posterity via <a href="https://tenforward.social/tags/wget" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#wget</a>. However, I also wanted to grab snapshots of any outlinks (of which there are many, as citations/references). By default, I couldn't figure out a configuration where wget would do that out of the box, without endlessly, recursively spidering the whole internet. I ended up making a kind-of poor man's <a href="https://tenforward.social/tags/ArchiveBox" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ArchiveBox</a> instead:for i in $(cat others.txt) ; do dirname=$(echo "$i" | sha256sum | cut -d' ' -f 1) ; mkdir -p $dirname ; wget --span-hosts --page-requisites --convert-links --backup-converted --adjust-extension --tries=5 --warc-file="$dirname/$dirname" --execute robots=off --wait 1 --waitretry 5 --timeout 60 -o "$dirname/wget-$dirname.log" --directory-prefix="$dirname/" $i ; doneBasically, there's a list of bookmarks^W URLs in others.txt that I grabbed from the initial mirror of the website with some <a href="https://tenforward.social/tags/grep" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#grep</a> foo. I want to do as good of a mirror/snapshot of each specific URL as I can, without spidering/mirroring endlessly all over. So, I hash the URL, and kick off a specific wget job for it that will span hosts, but only for the purposes of making the specific URL as usable locally/offline as possible. I know from experience that this isn't perfect. But... it'll be good enough for my purposes. I'm also stashing a WARC file. Probably a bit overkill, but I figure it might be nice to have.<a href="https://tenforward.social/tags/RedSails" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#RedSails</a> <a href="https://tenforward.social/tags/archive" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#archive</a> <a href="https://tenforward.social/tags/archival" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#archival</a> <a href="https://tenforward.social/tags/archiving" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#archiving</a> <a href="https://tenforward.social/tags/warc" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#warc</a>

Claudine C :antifa:Brief usage notes on <a href="https://aus.social/tags/ArchiveBox" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ArchiveBox</a> and <a href="https://aus.social/tags/wget" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#wget</a> <a href="https://www.claudinec.net/posts/2025-02-08-web-archiving/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://www.claudinec.net/posts/2025-02-08-web-archiving/</a><a href="https://aus.social/tags/SafeguardingResearch" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SafeguardingResearch</a> <a href="https://aus.social/tags/DigitalPreservation" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#DigitalPreservation</a> <a href="https://aus.social/tags/WebArchiving" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#WebArchiving</a>

GNU/Linux.chHTTrack - Der Website DownloaderIn diesem Tutorial zeige ich dir, wie du ganze Websites mit HTTrack für den Offline-Zugriff speichern kannst. Egal, ob für die eigene Sicherung oder einfach zum Stöbern ohne Internet – ich zeige dir Schritt für Schritt, wie es funktioniert. <a href="https://social.anoxinon.de/tags/httrack" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#httrack</a> <a href="https://social.anoxinon.de/tags/Curl" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Curl</a> <a href="https://social.anoxinon.de/tags/wget" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#wget</a> <a href="https://social.anoxinon.de/tags/Website" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Website</a> <a href="https://social.anoxinon.de/tags/Linux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Linux</a><a href="https://gnulinux.ch/httrack-der-website-downloader" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://gnulinux.ch/httrack-der-website-downloader</a>

Snagburz<a href="https://techhub.social/tags/unix" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#unix</a> and <a href="https://techhub.social/tags/linux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#linux</a> tip: The page you try to read on the web is a PDF file, but it takes the <a href="https://techhub.social/tags/browser" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#browser</a> forever to display it? From a terminal download it using wget <url><a href="https://techhub.social/tags/wget" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#wget</a> <a href="https://techhub.social/tags/offline" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#offline</a> <a href="https://techhub.social/tags/pc" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#pc</a> <a href="https://techhub.social/tags/tech" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#tech</a> <a href="https://techhub.social/tags/shell" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#shell</a>

Kevin Karhan :verified:<a href="https://mastodon.social/@bagder" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@bagder</a> Problem with that is (besides occasional bugfixes), most people including myself would see <a href="https://infosec.space/tags/curl" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#curl</a> to be functionally complete and anything "nice to have" would be considered not worth the balooning in <a href="https://infosec.space/tags/complexity" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#complexity</a> and <a href="https://infosec.space/tags/size" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#size</a>. <ul><li>I mean, does <a href="https://curl.se/" rel="nofollow noopener noreferrer" target="_blank">curl</a> need to be able to do <a href="https://infosec.space/tags/BitTorrent" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#BitTorrent</a> (magnet:), <a href="https://infosec.space/tags/IPFS" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#IPFS</a> (ipfs://) or god forbid <a href="https://infosec.space/tags/blockchain" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#blockchain</a> (i.e. <a href="https://infosec.space/tags/EVM" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#EVM</a>) support? </li><li>Do you really want to integrate <a href="https://mastodon.social/@torproject" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@torproject</a> / <a href="https://infosec.space/tags/Tor" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Tor</a> support natively into curl when using <a href="https://infosec.space/tags/HTTP" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#HTTP</a> (localhost:8118) and <a href="https://infosec.space/tags/SOCKS5" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SOCKS5</a> (localhost:9050) <a href="https://infosec.space/tags/proxy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#proxy</a> allows for the same and doesn't necessitate having to handle and ingest Tor arguments as well??</li></ul>In fact if <a href="https://infosec.space/tags/toybox" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#toybox</a> didn't have a <a href="https://infosec.space/tags/wget" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#wget</a> implementation that I could use for OS/1337 I would've merely chosen <a href="https://curl.se/tiny/" rel="nofollow noopener noreferrer" target="_blank"><code>tiny-curl -o</code></a> as a <a href="https://stackoverflow.com/questions/30702803/set-aliases-globally-for-all-users" rel="nofollow noopener noreferrer" target="_blank">global alias</a> or if <a href="https://infosec.space/tags/tinycurl" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#tinycurl</a> wasn't an option, <a href="https://linux.die.net/man/1/curl" rel="nofollow noopener noreferrer" target="_blank"><code>curl -o</code></a> instead.<ul><li>Maybe someone who wants to have said functionality like <code>tor</code> support built-in will go and IDK make i.e. <code>#neocurl</code> or sth. along those lines or build something like <code>#ethcurl</code> or <code>#torcurl</code>or <code>#ipfscurl</code> or whatever...</li></ul>That being said I am glad <code>curl</code> isn't solely maintained by you but has other contributors (give them a shoutout!) but I also am glad you maintain that vital software that most "<a href="https://infosec.space/tags/TechIlliterate" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#TechIlliterate</a> <a href="https://infosec.space/tags/Normies" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Normies</a>" most likely never heard of but propably use on a daily basis as part of all the <a href="https://infosec.space/tags/tech" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#tech</a> they use to <a href="https://infosec.space/tags/consume" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#consume</a> media with...<ul><li>I consider curl to be "the <a href="https://infosec.space/tags/vim" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#vim</a> of downloaders" (tho that's kinda insulting and limiting since <code>curl</code> is more than just a downloader and more intuitive than <code>vim</code>) with wget being "the <a href="https://infosec.space/tags/vi" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#vi</a> of downloaders" (tho <code>wget</code> is even simpler to use than <code>vi</code>)...</li></ul>Either way, curl is awesome... <a href="https://infosec.space/tags/OS1337" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#OS1337</a> <a href="https://infosec.space/tags/Enshittification" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Enshittification</a> <a href="https://infosec.space/tags/Bloat" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Bloat</a> <a href="https://infosec.space/tags/Bloatware" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Bloatware</a>

Thoralf Will 🇺🇦🇮🇱Jetzt habe ich das Ding zugreifbar. Ich kann mich an der Webschnittstelle auch anmelden.Interessanterweise funktioniert der Zugriff mittels <a href="https://soc.umrath.net/tags/curl" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#curl</a> (401) nicht (und alle Scripten, die darauf basieren schlagen fehlt) - während ich mit <a href="https://soc.umrath.net/tags/wget" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#wget</a> ein 200 und sinnvollen payload bekomme.curl -v -k -u XXX:YYY https://192.../startwget --no-check-certificate -v --user=XXX --password=YYY https://192.../startKann mir mal jemand über die Straße helfen?

dusoftAlso: it must be really a gargantuan task to do a simple GET to fetch followers, right? Neither <a href="https://fosstodon.org/tags/curl" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#curl</a>, nor <a href="https://fosstodon.org/tags/wget" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#wget</a> can do that, right? Right? <a href="https://fosstodon.org/tags/fediverse" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#fediverse</a> <a href="https://fosstodon.org/tags/mastodon" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#mastodon</a> <a href="https://fosstodon.org/tags/mastoadmin" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#mastoadmin</a> <a href="https://fosstodon.org/tags/MastoLivre" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#MastoLivre</a> <a href="https://fosstodon.org/tags/webdev" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#webdev</a>

daniel:// stenberg://I bet you don't remember when <a href="https://mastodon.social/tags/wget" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#wget</a> briefly considered switching to using <a href="https://mastodon.social/tags/libcurl" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#libcurl</a> back in 2007.<a href="https://daniel.haxx.se/blog/2007/10/27/wget-going-libcurl/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://daniel.haxx.se/blog/2007/10/27/wget-going-libcurl/</a>

don Elías (como los buses) 🥨<a href="https://www.impulsait.com/@impulsait.com" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@impulsait.com</a> esto es un comentario desde Mastodon.CR en el blog de <a href="https://www.impulsait.com/@impulsait.com" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@impulsait.com</a>La publicación de blog es sobre cómo usar <a href="https://mastodon.cr/tags/wget" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#wget</a> para hacer una copia fuera de línea de un sitio web, especialmente útil para convertir un sitio dinámico en estático por temas de rendimiento y seguridad

Impulsa IT SolutionsHacer una copia o espejo de un sitio usando wgetA veces uno necesita hacer una copia completa de un sitio web para acceso fuera de línea, o también para convertir un sitio dinámico a estático, especialmente cuando un cliente tiene un sitio web en una versión obsoleta de un manejador de contenido como WordPress o Joomla en una versión muy vieja y corriendo una versión de PHP vulnerable. Esto es especialmente útil cuando el sitio web no requiere ser modificado. Esto trae como beneficio un sitio más veloz y más seguro, ya que un sitio estático HTML se envía tal cual al navegador de las personas usuarias, sin tener que ejecutar código PHP potencialmente vulnerable.Con la herramienta de línea de comandos <code>wget</code> es posible de forma muy sencilla descargar un sitio web entero:<pre><code>wget --mirror --convert-links --adjust-extension --page-requisites --no-parent http://example.org</code></pre>Explicación de algunos de los argumentos:<ul><li><code>--mirror</code> – Descarga recursivamente y hace otros ajustes relacionados, implica: <code>-r -N -l inf --no-remove-listing</code>.</li><li><code>--convert-links</code> – convierte enlaces y referencias en HTML como CSS, para poder ser vistas fuera de línea.</li><li><code>--adjust-extension</code> – Ajusta la extensión de los objetos html o css según el tipo de datos</li><li><code>--page-requisites</code> – Descarga objetos enlazados como hojas de estilo CSS e imágenes requeridos para ver el sitio fuera de línea.</li><li><code>--no-parent</code> – No subir a directorios superiores en el URL, útil cuando se está haciendo espejo de un URL específico como <a href="http://example.org/sub-directorio/" rel="nofollow noopener noreferrer" translate="no" target="_blank">http://example.org/sub-directorio/</a></li></ul>Ejemplo de conversión de sitio WordPress en un sistema cPanel:<pre><code># Entrar como usuario a shell, para no ejecutar comandos como rootsu - user -s /bin/bash# Crear directorio de descargasmkdir download && cd downloadwget --mirror --convert-links --adjust-extension --page-requisites --no-parent http://example.org# Hacer un respaldo del directorio raíz del sitio webcd ..mv public_html public_html~# Colocar la copia descargada en líneamv download/example.org public_html# Probar sitio, ajustar detalles y opcionalmente borrar base de datos luego de hacer respaldo# Con esto el sitio dinámico ha sido convertido a estático.</code></pre>Referencias:<ul><li>GNU wget manual: <a href="https://www.gnu.org/software/wget/manual/wget.html" rel="nofollow noopener noreferrer" target="_blank">https://www.gnu.org/software/wget/manual/wget.html </a></li><li>Make Offline Mirror of a Site using wget: <a href="https://www.guyrutenberg.com/2014/05/02/make-offline-mirror-of-a-site-using-wget/" rel="nofollow noopener noreferrer" target="_blank">https://www.guyrutenberg.com/2014/05/02/make-offline-mirror-of-a-site-using-wget/</a></li></ul><a rel="nofollow noopener noreferrer" class="hashtag u-tag u-category" href="https://www.impulsait.com/tag/linux/" target="_blank">#Linux</a> <a rel="nofollow noopener noreferrer" class="hashtag u-tag u-category" href="https://www.impulsait.com/tag/mirror/" target="_blank">#mirror</a> <a rel="nofollow noopener noreferrer" class="hashtag u-tag u-category" href="https://www.impulsait.com/tag/static-site/" target="_blank">#staticSite</a> <a rel="nofollow noopener noreferrer" class="hashtag u-tag u-category" href="https://www.impulsait.com/tag/sysadmin/" target="_blank">#sysadmin</a> <a rel="nofollow noopener noreferrer" class="hashtag u-tag u-category" href="https://www.impulsait.com/tag/tips/" target="_blank">#tips</a> <a rel="nofollow noopener noreferrer" class="hashtag u-tag u-category" href="https://www.impulsait.com/tag/wget/" target="_blank">#wget</a>

Kevin Karhan :verified:<a href="https://equestria.social/@SweetAIBelle" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@SweetAIBelle</a> I just hope that <a href="https://mstdn.jp/@landley" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@landley</a> can do <a href="https://infosec.space/tags/alias" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#alias</a> support in <code>toysh</code> so I can supplement most stuff easily with a <code>.bash_aliases</code> <a href="https://github.com/kkarhan/misc-scripts/blob/master/bash/.bash_aliases" rel="nofollow noopener noreferrer" target="_blank">file</a> to make it easy to use...Like I know I eon't be abke to ship most tools that I want to have at hand with like a 1440kB image but I can just yoink them with <a href="https://infosec.space/tags/wget" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#wget</a> or <a href="https://infosec.space/tags/curl" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#curl</a> or <a href="https://infosec.space/tags/tinycurl" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#tinycurl</a> or whatever even before any <a href="https://github.com/os-1337/spm" rel="nofollow noopener noreferrer" target="_blank">package manager</a> is up and running or available.<ul><li>Tho compared to most <a href="https://infosec.space/tags/Linux" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Linux</a> distros and tools like <code>apt</code> and <code>yum</code>, <code>spm</code> isn't very advanced and at most will likely only offer to download/remove a binary or replace it...</li></ul>

Ƿynnie Ƿynn :fedora: :freebsd:How do you all pronounce <code>wget</code>? - double-u-get - wuh-get - we-get <a href="https://misskey.winscloud.net/tags/wget" rel="nofollow noopener noreferrer" target="_blank">#wget</a> <a href="https://misskey.winscloud.net/tags/linux" rel="nofollow noopener noreferrer" target="_blank">#linux</a>

Recent searches

Search options

Administered by:

Server stats:

#wget