Mastodon.green

0 posts0 participants0 posts today

st3fanDoes anybody still host and use a YaCy search engine instance? <a href="https://yacy.net/" rel="nofollow noopener noreferrer" target="_blank">https://yacy.net/</a>I love the concept, but it did stress my servers every time after a few weeks.My newest VPS has more of everything. I will give it another try.<a href="https://fedi.caliandroid.de/tags/yacy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#yacy</a> <a href="https://fedi.caliandroid.de/tags/searchengine" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#searchEngine</a>

Widowild :mastodon: :verified:<a href="https://mastodon.green/@fabimoria" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@fabimoria</a> salut, tu as <a href="https://mastodon.xyz/tags/SearXNG" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SearXNG</a> <a href="https://mastodon.xyz/tags/yacy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#yacy</a> <a href="https://mastodon.xyz/tags/mojeek" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#mojeek</a> <a href="https://mastodon.xyz/tags/kagi" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#kagi</a>

Kolibri**Проблема: Ограниченная доступность децентрализованных поисковых решений на основе YaCy в Gentoo****1. Децентрализация vs Централизованные поисковики** Большинство пользователей привыкли к централизованным поисковым системам (Google, Bing, Yandex), которые контролируют индексацию, фильтрацию контента и ранжирование. YaCy предлагает децентрализованный подход, но его популярность остаётся низкой из-за ряда технических и пользовательских барьеров. **2. Проблемы установки и совместимости в Gentoo** Gentoo известен своей гибкостью, но установка YaCy на этой платформе может быть сложной из-за: Отсутствия официального ebuild в основном репозитории. Потенциальных зависимостей, конфликтующих с текущими сборками. Отсутствия подробной документации для интеграции с системой. **3. Ограниченная функциональность и удобство для конечного пользователя** Хотя YaCy мощен с точки зрения приватности и автономности, он сталкивается с проблемами: Высокие требования к ресурсам при индексировании. Медленная скорость поиска при малом числе узлов. Ограниченные механизмы фильтрации контента по сравнению с традиционными поисковиками. **4. Интеграция в экосистему RuTracker.org** На форумах вроде RuTracker.org востребованы альтернативные поисковые решения. Однако: YaCy не всегда эффективно индексирует динамический контент форумов. Необходима дополнительная настройка парсеров для корректного сбора данных. Малое количество узлов, ориентированных на индексирование RuTracker, снижает качество поиска. **Вывод** YaCy в Gentoo и его потенциальная интеграция с RuTracker.org требуют более удобных инструментов развертывания, оптимизации индексации и повышения удобства работы для конечных пользователей. **Дополнительная база знаний для изучения и исследования** **1. Официальные ресурсы YaCy** Официальный сайт YaCy – документация, исходный код и последние обновления. GitHub YaCy – основной репозиторий проекта, баг-трекер, pull requests. Форум поддержки YaCy – обсуждения, вопросы и ответы. **2. Документация и исследования по децентрализованным поисковикам** DHT (Distributed Hash Table) и его применение в P2P-системах Сравнение децентрализованных поисковиков: SearX, YaCy, Whoogle Peer-to-Peer Search Engines: Opportunities and Challenges (ACM Digital Library) **3. Gentoo и его экосистема** Официальная документация Gentoo – руководство по установке и настройке пакетов. Bugzilla Gentoo – поиск и обсуждение ошибок, возможное добавление ebuild для YaCy. GURU overlay – сообщество разработчиков, вносящих новые пакеты.**Библиография** Callan, J. (2000). *Distributed Information Retrieval*. Springer. Balakrishnan, H., Kaashoek, M. F., Karger, D., Morris, R., & Stoica, I. (2003). *Looking up data in P2P systems*. Communications of the ACM, 46(2), 43-48. Stoica, I., Morris, R., Karger, D., Kaashoek, M. F., & Balakrishnan, H. (2001). *Chord: A scalable peer-to-peer lookup service for internet applications*. ACM SIGCOMM Computer Communication Review, 31(4), 149-160. Benzmüller, C., & Heyer, G. (2008). *Peer-to-peer information retrieval: An overview*. Springer. Gentoo Linux Wiki (2024). *Installing and Configuring Packages in Gentoo*. Retrieved from <a href="https://wiki.gentoo.org" rel="nofollow noopener noreferrer" target="_blank">https://wiki.gentoo.org</a>. YaCy Developers (2023). *YaCy Search Engine: Architecture and Performance Optimization*. Retrieved from <a href="https://github.com/yacy/yacy_search_server" rel="nofollow noopener noreferrer" target="_blank">https://github.com/yacy/yacy_search_server</a>.**Хэштеги** <a href="https://qoto.org/tags/YaCy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#YaCy</a> <a href="https://qoto.org/tags/DecentralizedSearch" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#DecentralizedSearch</a> <a href="https://qoto.org/tags/Gentoo" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Gentoo</a> <a href="https://qoto.org/tags/RuTracker" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#RuTracker</a> <a href="https://qoto.org/tags/P2P" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#P2P</a> <a href="https://qoto.org/tags/DistributedSearch" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#DistributedSearch</a> <a href="https://qoto.org/tags/DHT" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#DHT</a> <a href="https://qoto.org/tags/FOSS" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#FOSS</a> <a href="https://qoto.org/tags/PrivacyTech" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#PrivacyTech</a> <a href="https://qoto.org/tags/PeerToPeer" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#PeerToPeer</a> <a href="https://qoto.org/tags/OpenSource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#OpenSource</a>**Где найти соратников для обсуждения?** 🔹 **Официальные сообщества YaCy** Форум YaCy Community Группа в Matrix: <a href="https://qoto.org/tags/yacy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#yacy</a>:matrix.org IRC-канал: <a href="https://qoto.org/tags/yacy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#yacy</a> на irc.libera.chat 🔹 **Сообщества по Gentoo и Open Source** Форум Gentoo Reddit: r/Gentoo Telegram-группа Gentoo Russia 🔹 **Дискуссионные площадки по децентрализованным технологиям** LOR (Linux.org.ru) – обсуждение Linux и open-source решений. RuTracker.org – форум альтернативных технологий Hacker News – обсуждение перспектив P2P и децентрализованных систем. Эти ресурсы помогут разработчикам, исследователям и энтузиастам YaCy глубже разобраться в технологии и найти единомышленников.<a href="https://matrix.to/#/!NggrnptZjGBkegXXqK:matrix.org/$-2Ue3xSsaSzVX1voxfefOFN1PvQT6gAUAVJxM2Hfsyk?via=matrix.org&via=monerobay.com" rel="nofollow noopener noreferrer" target="_blank">https://matrix.to/#/!NggrnptZjGBkegXXqK:matrix.org/$-2Ue3xSsaSzVX1voxfefOFN1PvQT6gAUAVJxM2Hfsyk?via=matrix.org&via=monerobay.com</a>

SantiagoEstoy haciendo un experimento con YaCy, un buscador p2p para indexar internet. Que sitios interesantes para escanear se les ocurren? Sitios que tengan info sin tener que loguearse, como bibilotecas, tutoriales, manuales, enciclopedias, conocimiento, tecnología, cultura, atre, literatura, etc. Comenten que enlaces les parecen importantes asi los voy agregando a la lista de crawl, quiero ver que se puede lograr. Monte un servidor dedicado exclusivamente a esto, a escanear internet, es medio un delirio, pero es tanta la basura que me tiran los disquebuscadores que me parece que me voy a montar el mio propio <a href="https://mastodon.uy/tags/yacy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#yacy</a> <a href="https://mastodon.uy/tags/p2p" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#p2p</a> <a href="https://mastodon.uy/tags/buscadores" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#buscadores</a> <a href="https://mastodon.uy/tags/search" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#search</a> <a href="https://mastodon.uy/tags/engine" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#engine</a> <a href="https://mastodon.uy/tags/internet" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#internet</a> <a href="https://mastodon.uy/tags/undernet" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#undernet</a>

klokanekWhat are <a href="https://witter.cz/tags/YaCy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#YaCy</a> users asking YaCy about YaCy?<a href="https://community.searchlab.eu/t/real-yacy-faq-frequently-asked-queries/1756/2" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://community.searchlab.eu/t/real-yacy-faq-frequently-asked-queries/1756/2</a><a href="https://witter.cz/tags/search" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#search</a> <a href="https://witter.cz/tags/p2p" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#p2p</a>

Hieronymus<a href="https://chaos.social/tags/digital" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#digital</a> <a href="https://chaos.social/tags/tinker" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#tinker</a>|ing° tweaking <a href="https://chaos.social/tags/yacy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#yacy</a> ° re-enabling <a href="https://chaos.social/tags/keyfile" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#keyfile</a> access to my server ° re-enabling my homepage ° rescue my old <a href="https://chaos.social/tags/NextCloud" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#NextCloud</a> instance

rwa<a href="https://infosec.space/@kkarhan" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@kkarhan</a> <a href="https://swiss.social/@thom" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@thom</a> <a href="https://suma-ev.social/@osf" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@osf</a> <a href="https://suma-ev.social/@openwebsearcheu" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@openwebsearcheu</a> <a href="https://suma-ev.social/@MetaGer" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@MetaGer</a> <a href="https://monocles.social/@monocles" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@monocles</a> aber ist das dann nicht auch "nur" wieder eine Meta-Suchmaschine die andere Suchindizes nutzt? So kannte ich <a href="https://clttr.it/tags/searx" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#searx</a> bisher zumindest. Der einzige mir bekannte distributed Suchindex basiert auf <a href="https://clttr.it/tags/yacy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#yacy</a> <a href="https://yacy.net/" rel="nofollow noopener noreferrer" target="_blank">https://yacy.net/</a>

Alterego_MidshipmanA reference server project for researching the possibilities of moving YaCy to an alternative codebase Introduction YaCy is an open source decentralized search engine written in Java. However, its performance and scalability are limited by the current selection of technologies. This project involves the creation of a reference server for testing the transition of YaCy to more efficient programming languages, such as C++, C, Rust and Go. Goal Productivity and efficiency research alternative languages in the development of a decentralized search system. Optimization of the use of hardware resources, including multiprocessor systems, large amounts of RAM and GRAID. Providing better multithreading support and expansion of search algorithm capabilities. Reduced dependency on the JVM to increase performance and reduce resource usage. Hardware platform The project involves the use of a reference server with the following characteristics: RAM: 1–10 TB (depending on configuration and indexing volume) Processors: 4-16 Intel server processors Co-processors: Nvidia graphics cards for processing large amounts of data Refuge: GRAID to increase data access speed and reduce latency Network interaction: Optimized network protocols for efficient exchange of information between nodes The main stages of development Analysis of the current YaCy architecture and identification of key limitations. Selection of the appropriate programming language (C++, C, Rust, Go) and testing of their capabilities. Development of a prototype of an alternative core of the search engine. Integration with existing infrastructure and performance testing on a reference server. Evaluation of efficiency and further YaCy is an open source decentralized search engine written in Java. However, its performance and scalability are limited by the current selection of technologies. This project involves the creation of a reference server for testing the transition of YaCy to more efficient programming languages, such as C++, C, Rust and Go.improvement of algorithms. Expected results Improved performance of the search engine due to the efficient use of multi-core processors and GRAID. Reduced RAM usage and increased indexing speed. Ensuring greater stability and scalability of the system. Possibility of further adaptation of the new code base for use in other decentralized projects. This project will become the basis for future improvements in the field of decentralized search systems and will demonstrate the possibility of creating a high-performance alternative to YaCy using modern technologies. A reference server project for deploying and exploring the possibilities of transitioning YaCy to the C++, C, Rust, Go code base with 1-10 TB RAM and GRAID, with 4-16 Intel processors and NVidia cards as co-processors . This is a very ambitious and exciting project! Here are some suggestions for server configuration and budgeting for such a study. Reference server configuration: RAM: 1–10 TB: Large amount of memory allows efficient processing and indexing of large amounts of data while maintaining low query latency. It is recommended to use servers with scalable memory support, e.g. DDR4 ECC RAM for stability. Processors (CPU): 4-16 Intel server processors: Multi-core processors, e.g. Intel Xeon or Intel Cascade Lake, which support large computing loads and high throughput. It is suggested to consider multi-processor systems, which allow parallel computations to be handled and improve overall performance. Graphics cards (GPU): Nvidia GPU to process large amounts of data, including usage DIFFERENT to speed up certain processes, such as data analysis or deep learning. Type cards are suitable for this Nvidia Tesla V100 or A100, which are specifically focused on large-scale computing and data processing. Storage (Storage): GRADE — technology for increasing storage performance. Using NVMe SSD in combination with RAID arrays helps minimize delays when accessing data. Terabytes of storage for indexing large volumes of web data. Networking: Optimized network protocols: use 10G Ethernet or InfiniBand for fast data transfer between nodes in the network and to support scalability. Technologies for development: Programming languages: C++/C: for the most efficient use of hardware resources, high performance and the possibility of manual memory management. Rust: due to its security and high efficiency, it will be an excellent choice for systems working with rich flows. Go: convenient for parallel computing and for building scalable network applications, which is well suited for decentralized systems. Comparing the performance of these languages will allow to determine the optimal one for further stages of development. Search and indexing algorithms: Determination of effective algorithms for processing large volumes of data. Development of algorithms for optimization of multi-threading and work with multi-core processors. Estimate: Equipment: Intel Xeon servers with 4-16 processors: around $10,000-$50,000 depending on the number of processors and the level of computing power. Nvidia Tesla GPU: approximately $5,000-$15,000 per card. RAM: 1 TB RAM — $10,000-$30,000. Refuge (NVMe SSD/GRAID): around $2,000-$10,000 for high performance solutions. Licenses and Software: Operating system: Free based solutions Linux (e.g. Ubuntu). Development tools: depending on the programming language, may be free (for example, for Rust, Go). Infrastructure and testing: Using cloud-based testing platforms can add about $1,000-$5,000 per year to the budget for additional capacity and monitoring tools. Expected results: Improved performance thanks to more efficient use of multi-core processors and high-performance GPUs. Reducing delays and resource utilization due to optimization of indexing and query processing. Scalability systems for processing big data and search queries. Open source and adaptation for other decentralized projects that can contribute to the development of technologies in the field of search engines. This is definitely a big project that can significantly affect the evolution of decentralized search engines. Here is a table that shows the maximum configuration of the reference server and a rough estimate for researching the transition of YaCy to alternative codebases (C++, C, Rust, Go): Component Maximum configuration Estimated cost Processors (CPU) 16 Intel Xeon server processors (Cascade Lake) 40,000–80,000 dollars RAM 10 TB DDR4 ECC RAM $100,000–$300,000 Graphics cards (GPU) 4 Nvidia Tesla V100 or A100 (for data processing and acceleration) 20,000–60,000 dollars Storage (Storage) 100 TB NVMe SSD + GRAID for high performance storage 20,000–40,000 dollars Network protocols 10G Ethernet or InfiniBand for fast data transfer $5,000–$15,000 Operating system Ubuntu or other free Linux distributions Free Development and testing tools Docker, Kubernetes, profiling and monitoring tools $1,000-$5,000 (Licenses, Tools) Power consumption and cooling Energy consumption for such systems including cooling $5,000-$10,000 per year General estimate (approximate) $190,000 to $510,000 Notes: Processors: The cost of Intel Xeon (Cascade Lake) server processors may vary by number of cores and speed, as well as by specific models. RAM: The cost of memory for servers with large amounts of memory (up to 10 TB) increases significantly, so the price may vary depending on the amount. Graphics cards (GPU): The cost of Nvidia Tesla depends on specific models. For the tasks of processing large volumes of data and acceleration, you can use models of the V100 or A100 type. Refuge: This amount of storage will require usage NVMe SSD in combination with GRADE, which will significantly improve the speed of data access. Network protocols: For high-speed data transmission between nodes, it is important to have a network infrastructure of the type 10G Ethernet or InfiniBand. This is an approximate estimate for the maximum configuration, based on typical server components for such projects. These costs may vary depending on specific hardware vendors and licenses. In order to implement a reference server to study the transition of YaCy to alternative codebases, it is important to develop a clear budget for the implementer, as well as a list of potential candidates who can be involved in the project. Estimate of the implementer: Stage Description Estimated cost Design and planning Development of the technical task, preparation of the project implementation plan 10,000–20,000 dollars Selection and purchase of equipment Evaluation and purchase of server equipment, network components, GPU 200,000–500,000 dollars Server settings Installation and configuration of servers, including network and data storage settings 30,000–50,000 dollars Development of a system core prototype Development of an initial prototype of an alternative YaCy kernel in new languages 40,000–80,000 dollars Integration with existing infrastructure Integrating the new code base with the current YaCy infrastructure 20,000–40,000 dollars Testing and monitoring Performance, stability and scalability tests 20,000–40,000 dollars Evaluation of results and optimization Evaluation of efficiency, introduction of optimizations and corrections of algorithms 10,000–20,000 dollars Documentation and training Preparation of documentation, staff training and project support 10,000–15,000 dollars Total estimated cost $340,000 to $765,000 List of candidates for implementation: Technical project manager (Project Manager) Responsible for overall project coordination, planning and organization of works. Requirements: experience in IT project management, knowledge of Agile or Waterfall methodologies, experience in managing large teams. System architect Develops the overall architecture of the server infrastructure, is responsible for optimization and scalability. Requirements: experience working with highly loaded systems, knowledge of C++, Rust, Go technologies, experience in building and configuring server solutions. Developers (C++, C, Rust, Go) A new search engine core is being developed in selected programming languages. Requirements: in-depth knowledge of relevant programming languages, experience in developing scalable applications, familiarity with decentralized systems and search algorithms. Server configuration engineer Responsible for physical setup and configuration of servers, including GRAID deployment, configuration of graphics cards and network components. Requirements: experience in configuring server systems and working with highly loaded infrastructures. Test Engineer (QA) Designs and runs tests to verify system performance, scalability, and reliability. Requirements: experience in testing distributed and highly loaded systems, knowledge of testing tools such as JMeter or other similar tools. Monitoring and optimization engineer Responsible for monitoring system performance, finding and eliminating performance bottlenecks. Requirements: experience in setting up system monitoring and profiling systems (Prometheus, Grafana, or similar), optimization of resource use. Documentation and technical writer Responsible for the preparation of technical documentation, description of algorithms, instructions for setting up and operating the system. Requirements: experience in writing technical documentation for complex IT projects. Total costs for the implementer: Team salary: Technical Manager: $80,000–120,000 per year System Architect: $90,000-$150,000 per year Developers: $70,000-$120,000 per year (each) Configuration and Test Engineers: $60,000-$100,000 per year (each) Documentation and technical writer: $50,000-$80,000 per year Implementation time: approximately 12–18 months (depending on the complexity and scope of the project). License and tool costs: Development and testing software: $5,000-$10,000 Licenses for server operating systems and monitoring tools: $10,000–20,000 This estimate is indicative and may vary depending on specific circumstances, suppliers and selected solutions for project implementation. When considering taxes in the EU, particularly in the Netherlands, it is necessary to include corporate income taxes, VAT (value added tax) and other possible financing costs. The Netherlands has one of the most transparent and efficient tax systems in the EU, but it is important to consider various aspects. Taxes in the Netherlands for IT projects: Corporate Income Tax (CIT): The income tax rate in the Netherlands is 19% for income up to €200,000 and 25.8% for income above this threshold. VAT: The standard VAT rate in the Netherlands is 21%. For certain types of goods and services, reduced rates are possible (for example, 9% for some goods and services). Social contributions: Salary expenses are subject to social contributions. This includes pension and medical contributions, which are approximately 27.65% of workers' wages. Taxes on dividends: A dividend tax of 15% is applied to the payment of dividends at the enterprise level. Other fees: The Netherlands has some additional fees for certain activities, including contributions to environmental initiatives or taxes on the use of specific resources. Accounting for taxes in the project estimate: Stage Estimated cost before taxes Tax expenses (approx.) Approximate cost with taxes Design and planning 10,000–20,000 dollars $2,000-$4,000 (20%) $12,000–$24,000 Selection and purchase of equipment 200,000–500,000 dollars $42,000-$105,000 (21%) $242,000 to $605,000 Server settings 30,000–50,000 dollars $6,300-$10,500 (21%) $36,300–$60,500 Development of a system core prototype 40,000–80,000 dollars $8,400-$16,800 (21%) $48,400–$96,800 Integration with infrastructure 20,000–40,000 dollars $4,200-$8,400 (21%) $24,200–$48,400 Testing and monitoring 20,000–40,000 dollars $4,200-$8,400 (21%) $24,200–$48,400 Evaluation of results and optimization 10,000–20,000 dollars $2,000-$4,000 (20%) $12,000–$24,000 Documentation and training 10,000–15,000 dollars $2,100-$3,150 (21%) $12,100-$18,150 The total cost of the project $340,000 to $765,000 $71,100-$161,850 $411,100 to $926,850 Additional costs for social contributions: If the company hires employees, social security contributions can further increase wage costs (approximately 27.65%). This applies to the hiring of both local specialists and foreign specialists if they work within the Netherlands. Estimated total cost with taxes for the project: Taking into account income taxes, VAT and other costs, the total cost of the project based on the maximum targeted costs will look like this: Approximate cost without taxes: $340,000 – $765,000 Estimated tax costs (approximately 21%): 71,100 - 161,850 dollars Approximate cost with taxes: 411,100 - 926,850 dollars These costs may vary depending on various factors, such as the exact VAT rates, the specifics of social contributions and possible changes in Dutch tax law. For project management and cost estimation, it is important to choose companies that specialize in technical consulting, large IT project management, and have experience in areas such as decentralized systems development, big data processing, and infrastructure scaling. Here are several categories of companies and a list of candidates that may be involved in such a project: Types of companies for project management: IT consulting and system integration: They specialize in the development, testing and integration of complex IT systems. They have experience in managing projects with large technical requirements and preparing relevant estimates. Infrastructure technology consulting companies: Support project teams in the development of scalable and high-performance solutions that require significant resources. Software development outsourcing company: Connecting specialists to specific tasks of developing the search engine core and optimizing performance. Possible candidates for project management: Accenture Description: Global consulting company with vast experience in the field of IT consulting, development and project management at the international level. Field of specialization: Development of large corporate solutions, optimization and scaling of infrastructure. Why choose: Has experience in working with large technological projects, such as large search engines and data processing. Get hold of it Description: A large international consulting company with strong positions in technology project management, including automation and digital transformation. Field of specialization: Software development, introduction of new technologies (including C++, Rust, Go). Why choose: He has experience in implementing technologies for large distribution and high-performance systems. IBM Global Services Description: A classic provider of services for the development and scaling of IT systems. Field of specialization: Software development, consulting on infrastructure optimization, including for processing large volumes of data. Why choose: He has extensive experience in infrastructure projects and the deployment of complex technologies. Tata Consultancy Services (TCS) Description: One of the leaders in the field of IT consulting, known for its powerful resources in the field of software development and infrastructure. Field of specialization: Consulting, software development, management of large projects, infrastructure support. Why choose: Extensive experience in creating and optimizing distribution and large systems. Wipro Description: An Indian outsourcing company with a global scale and experience in working with large projects in various industries. Field of specialization: Software development, testing, as well as strategic project management based on modern technologies. Why choose: Specializes in large IT projects, including the integration of various technologies for distribution systems. Deloitte Consulting Description: One of the largest consulting companies in the world, providing services in the field of IT consulting, finance, management consulting and strategic planning. Field of specialization: Project management, development of infrastructure and scalable solutions, optimization of resources. Why choose: He has experience in the implementation of technologies for large infrastructure and technological projects. List of candidates for project management positions: Technical Director (CTO): Candidates: Senior Solution Architect with experience in developing scalable distributed systems. Experienced engineer in optimizing productivity in working with big data. Project Manager (PM): Candidates: Experienced in managing large IT software development projects, including infrastructure and scaling. Software Architect: Candidates: Specialists with experience in C++, Rust, Go, Java for the development of search engines. Big Data and Infrastructure Engineers: Candidates: Engineers with experience in GRAID, distributed systems and big data processing. Testers and QA specialists: Candidates: Specialists in testing high-performance systems for big data. Cost of project manager services: Company Approximate cost of services (monthly) Note Accenture 50,000–150,000 euros High level of service for large projects with large-scale infrastructures. Get hold of it 40,000–120,000 euros Offers services for development and optimization of complex systems. IBM Global Services 60,000–200,000 euros Support in the development and integration of complex technologies. Wipro 30,000–100,000 euros Provision of complex software and infrastructure development services. Deloitte 70,000–180,000 euros Offers consulting for large transformational IT projects. These cost estimates may vary depending on the complexity of the project, specialization and region in which the company is located. Keywords: YaCy, P2P, search engine, C++, Rust, Go, GRAID, Intel, NVidia, co-processors, big computing, decentralized search. <a href="https://qoto.org/tags/YaCy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#YaCy</a> <a href="https://qoto.org/tags/P2P" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#P2P</a> <a href="https://qoto.org/tags/SearchEngine" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SearchEngine</a> <a href="https://qoto.org/tags/Cplusplus" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Cplusplus</a> <a href="https://qoto.org/tags/Rust" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Rust</a> <a href="https://qoto.org/tags/GoLang" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GoLang</a> <a href="https://qoto.org/tags/GRAID" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GRAID</a> <a href="https://qoto.org/tags/Intel" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Intel</a> <a href="https://qoto.org/tags/Nvidia" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Nvidia</a> <a href="https://qoto.org/tags/Decentralized" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Decentralized</a> <a href="https://qoto.org/tags/BigData" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#BigData</a> <a href="https://qoto.org/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://qoto.org/tags/OpenSource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#OpenSource</a> <a href="https://qoto.org/tags/Networking" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Networking</a>

Pure Acetone💡 **YaCy + IPFS: Децентрализованный поиск для децентрализованного интернета** 🌐🔍 Что если объединить YaCy и IPFS, создав полностью независимый поисковик без цензуры и контроля корпораций? 🤔 🔹 Индексация IPFS-контента через DHT 🔹 Полнотекстовый поиск по CID и метаданным 🔹 P2P-хранение индексов в IPLD 🔹 DAO-управление и криптоэкономика ⚡Теория? Пока да. Потенциал? Огромный. Вопрос реализации — времени и сообщества. <a href="https://qoto.org/tags/YaCy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#YaCy</a> <a href="https://qoto.org/tags/IPFS" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#IPFS</a> <a href="https://qoto.org/tags/Decentralization" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Decentralization</a> <a href="https://qoto.org/tags/P2P" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#P2P</a> <a href="https://qoto.org/tags/Web3" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Web3</a> <a href="https://qoto.org/tags/OpenSource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#OpenSource</a> <a href="https://qoto.org/tags/SearchEngine" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SearchEngine</a> <a href="https://qoto.org/tags/CyberpunkTechYaCy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#CyberpunkTechYaCy</a> + IPFS: Децентрализированный поиск для децентрализованного интернета: <a href="https://orwellboxxx4.blogspot.com/2025/02/yacy-ipfs.html" rel="nofollow noopener noreferrer" target="_blank">https://orwellboxxx4.blogspot.com/2025/02/yacy-ipfs.html</a>

Pure Acetone**Описание статьи: YaCy + IPFS: Децентрализированный поиск для децентрализованного интернета** **📌 Основная идея** Статья исследует перспективы интеграции YaCy — децентрализованной поисковой системы — с IPFS, распределенной файловой сетью. Такое сочетание могло бы устранить проблему поиска в IPFS и создать альтернативу централизованным поисковым системам. **🔍 Основные темы** - Проблема поиска в децентрализованном интернете - Как работает YaCy и ее P2P-индексация - Возможные механизмы интеграции с IPFS - Потенциальные вызовы и решения - Перспективы и сроки реализации **📈 Актуальные обновления и трекинг темы** В статье регулярно обновляются данные о разработках в области децентрализованных поисковых систем, IPFS и возможных аналогах. Следите за новостями, исследованиями и техническими экспериментами по этой теме. **📎 Ссылка на статью:** [YaCy + IPFS: Децентрализированный поиск для децентрализованного интернета](<a href="https://orwellboxxx4.blogspot.com/p/yacy-ipfs.html" rel="nofollow noopener noreferrer" target="_blank">https://orwellboxxx4.blogspot.com/p/yacy-ipfs.html</a>) **<a href="https://qoto.org/tags/YaCy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#YaCy</a> <a href="https://qoto.org/tags/IPFS" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#IPFS</a> <a href="https://qoto.org/tags/P2P" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#P2P</a> <a href="https://qoto.org/tags/Web3" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Web3</a> <a href="https://qoto.org/tags/Decentralization" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Decentralization</a> <a href="https://qoto.org/tags/OpenSource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#OpenSource</a> <a href="https://qoto.org/tags/DHT" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#DHT</a> <a href="https://qoto.org/tags/%D0%94%D0%B5%D1%86%D0%B5%D0%BD%D1%82%D1%80%D0%B0%D0%BB%D0%B8%D0%B7%D0%B0%D1%86%D0%B8%D1%8F" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Децентрализация</a> <a href="https://qoto.org/tags/%D0%9F%D0%BE%D0%B8%D1%81%D0%BA" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Поиск</a> <a href="https://qoto.org/tags/%D0%A2%D0%B5%D1%85%D0%BD%D0%BE%D0%BB%D0%BE%D0%B3%D0%B8%D0%B8%D0%91%D1%83%D0%B4%D1%83%D1%89%D0%B5%D0%B3%D0%BE" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ТехнологииБудущего</a>**

michaelSet up a Yacy search! P2P searching is now something my little server is helping out with. Its interesting, it knows both more and less than google/bing/etc... I played around with it years and years ago. I like the admin panel...even though I dont really understand all of what is going on. Im going read some docs :). <a href="https://social.chrisco.me/tags/selfhost" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#selfhost</a> <a href="https://social.chrisco.me/tags/yacy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#yacy</a>

Michael Christen / YaCy SearchDietPi (lightweight Debian OS for SBCs) comes with <a href="https://sigmoid.social/tags/yacy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#yacy</a> <a href="https://dietpi.com/docs/software/distributed_projects/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://dietpi.com/docs/software/distributed_projects/</a>

Michael Christen / YaCy SearchMit <a href="https://sigmoid.social/tags/YaCy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#YaCy</a> auf den <a href="https://sigmoid.social/tags/CLT2025" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#CLT2025</a> / Chemnitzer Linuxtage - Blog Post von Frank: <a href="https://do3eet.pages.dev/post/clt2025yacy/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://do3eet.pages.dev/post/clt2025yacy/</a>

HieronymusThat escalated quickly. My <a href="https://chaos.social/tags/yacy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#yacy</a> folder is now bigger than my media library.

🌈 Lascapi ⁂Salut, je fais un test de référencement pour <a href="https://mastodon.zaclys.com/tags/yacy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#yacy</a> J’ai simplement fait une page html, bien statique, avec le mot yacy partout pour voir comment c’est géré dans les résultats. <a href="https://mastodon.zaclys.com/tags/yacynet" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#yacynet</a> <a href="https://mastodon.zaclys.com/tags/websearch" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#websearch</a> <a href="https://mastodon.zaclys.com/tags/resilience" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#resilience</a> <a href="https://mastodon.zaclys.com/tags/manipulation" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#manipulation</a> <a href="https://yacy.isman.fr/" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://yacy.isman.fr/</a>

Ella Sola<a href="https://sociale.network/@domanipagani" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@domanipagani</a> <a href="https://mastodon.uno/@filippodb" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@filippodb</a> <a href="https://mastodon.social/@dansup" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@dansup</a> sono andata a cercare info su <a href="https://mastodon.social/tags/duckduckgo" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#duckduckgo</a> e ho avuto una cattiva sorpresa. Sia quella che <a href="https://mastodon.social/tags/ecosia" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#ecosia</a> usano Bing 😖 cioè Microsoft 🤮 . Quindi alla fine contribuiamo ancora con il gigante USA. A quanto pare le uniche open source / libere di rapporti con le grandi aziende / che non raccolgono dati e rispettano la privacy / progettate nella UE , sono <a href="https://mastodon.social/tags/SearNXG" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SearNXG</a> e <a href="https://mastodon.social/tags/YaCy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#YaCy</a> . Ma SearNXG è più " prestante " e semplice da usare ( secondo ciò che dicono, ancora non ho provato.

Pure Acetone**Введение** YaCy — это децентрализованная поисковая система с открытым кодом, написанная на Java. Однако её производительность и возможности масштабирования ограничены текущим выбором технологий. Этот проект предполагает создание референс-сервера для тестирования перехода YaCy на более эффективные языки программирования, такие как C++, C, Rust и Go.<a href="https://pocketnet.app/index?s=7966712763dcebac145c433264ef74f5132e297a914c302a48e1652821002441&ref=PDQmXtWJfd8f6fMtwaxzCuqvLpoN3KbS4o" rel="nofollow noopener noreferrer" target="_blank">https://pocketnet.app/index?s=7966712763dcebac145c433264ef74f5132e297a914c302a48e1652821002441&ref=PDQmXtWJfd8f6fMtwaxzCuqvLpoN3KbS4o</a>**Цель** - Исследование производительности и эффективности альтернативных языков в разработке децентрализованных поисковых систем. - Оптимизация использования аппаратных ресурсов, включая многопроцессорные системы, большие объёмы оперативной памяти и GRAID. - Обеспечение лучшей поддержки многозадачности и расширение возможностей поискового алгоритма. - Снижение зависимости от JVM для повышения быстродействия и уменьшения использования ресурсов. **Аппаратная платформа** Проект включает использование референс-сервера со следующими характеристиками: - **Оперативная память**: 1–10 ТБ (в зависимости от конфигурации и объёма индексации) - **Процессоры**: 4–16 серверных процессоров Intel - **Ко-процессоры**: Графические карты Nvidia для обработки больших объёмов данных - **Хранение**: GRAID для повышения скорости доступа к данным и снижения задержек - **Сетевое взаимодействие**: Оптимизированные сетевые протоколы для эффективного обмена данными между узлами **Этапы разработки** 1. Анализ текущей архитектуры YaCy и определение ключевых ограничений. 2. Выбор подходящего языка программирования (C++, C, Rust, Go) и тестирование их возможностей. 3. Разработка прототипа альтернативного ядра поисковой системы. 4. Интеграция с существующей инфраструктурой и тестирование производительности на референс-сервере. 5. Оценка эффективности и дальнейшее улучшение алгоритмов. **Ожидаемые результаты** - Улучшенная производительность поисковой системы за счет эффективного использования многозадачных процессоров и GRAID. - Снижение использования оперативной памяти и повышение скорости индексации. - Обеспечение большей стабильности и масштабируемости системы. - Возможность дальнейшей адаптации новой кодовой базы для использования в других децентрализованных проектах. Этот проект станет основой для будущих улучшений в области децентрализованных поисковых систем и продемонстрирует возможность создания высокопроизводительной альтернативы YaCy с использованием современных технологий. **Проект референс-сервера для развертывания и исследования возможностей перехода YaCy на кодовую базу C++, C, Rust, Go с оперативной памятью 1-10 ТБ и GRAID, с 4-16 процессорами Intel и графическими картами Nvidia в качестве ко-процессоров.** **Ключевые слова**: YaCy, P2P, поисковая система, C++, Rust, Go, GRAID, Intel, Nvidia, ко-процессоры, большие вычисления, децентрализованный поиск. <a href="https://qoto.org/tags/YaCy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#YaCy</a> <a href="https://qoto.org/tags/P2P" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#P2P</a> <a href="https://qoto.org/tags/SearchEngine" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SearchEngine</a> <a href="https://qoto.org/tags/Cplusplus" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Cplusplus</a> <a href="https://qoto.org/tags/Rust" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Rust</a> <a href="https://qoto.org/tags/GoLang" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GoLang</a> <a href="https://qoto.org/tags/GRAID" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GRAID</a> <a href="https://qoto.org/tags/Intel" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Intel</a> <a href="https://qoto.org/tags/Nvidia" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Nvidia</a> <a href="https://qoto.org/tags/Decentralized" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Decentralized</a> <a href="https://qoto.org/tags/BigData" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#BigData</a> <a href="https://qoto.org/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://qoto.org/tags/OpenSource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#OpenSource</a> <a href="https://qoto.org/tags/Networking" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Networking</a>

Pure Acetone**Introduction** YaCy is a decentralized search engine with open-source code, written in Java. However, its performance and scalability are limited by the current choice of technologies. This project aims to create a reference server to test the transition of YaCy to more efficient programming languages, such as C++, C, Rust, and Go. **Objective** - Research the performance and efficiency of alternative languages in the development of decentralized search engines. - Optimize the use of hardware resources, including multi-core systems, large memory capacities, and GRAID. - Provide better multi-threading support and expand the search algorithm capabilities. - Reduce dependency on JVM to increase speed and minimize resource usage. **Hardware Platform** The project involves using a reference server with the following specifications: - **RAM**: 1–10 TB (depending on configuration and indexing volume) - **Processors**: 4–16 Intel server processors - **Co-processors**: Nvidia graphics cards for processing large data volumes - **Storage**: GRAID to improve data access speed and reduce latency - **Network Interaction**: Optimized network protocols for efficient data exchange between nodes **Development Stages** 1. Analyze the current YaCy architecture and identify key limitations. 2. Select suitable programming languages (C++, C, Rust, Go) and test their capabilities. 3. Develop a prototype for an alternative search engine core. 4. Integrate with the existing infrastructure and test performance on the reference server. 5. Evaluate efficiency and further improve algorithms. **Expected Outcomes** - Enhanced search engine performance through the effective use of multi-core processors and GRAID. - Reduced RAM usage and increased indexing speed. - Greater system stability and scalability. - The potential for further adaptation of the new codebase for use in other decentralized projects. This project will lay the foundation for future improvements in decentralized search systems and demonstrate the possibility of creating a high-performance alternative to YaCy using modern technologies. **Project of a reference server for deployment and exploration of the transition of YaCy to C++, C, Rust, Go codebase with 1-10 TB of RAM and GRAID, 4-16 Intel processors, and Nvidia graphics cards as co-processors.** **Keywords**: YaCy, P2P, search engine, C++, Rust, Go, GRAID, Intel, Nvidia, co-processors, big computing, decentralized search. <a href="https://qoto.org/tags/YaCy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#YaCy</a> <a href="https://qoto.org/tags/P2P" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#P2P</a> <a href="https://qoto.org/tags/SearchEngine" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SearchEngine</a> <a href="https://qoto.org/tags/Cplusplus" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Cplusplus</a> <a href="https://qoto.org/tags/Rust" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Rust</a> <a href="https://qoto.org/tags/GoLang" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GoLang</a> <a href="https://qoto.org/tags/GRAID" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GRAID</a> <a href="https://qoto.org/tags/Intel" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Intel</a> <a href="https://qoto.org/tags/Nvidia" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Nvidia</a> <a href="https://qoto.org/tags/Decentralized" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Decentralized</a> <a href="https://qoto.org/tags/BigData" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#BigData</a> <a href="https://qoto.org/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://qoto.org/tags/OpenSource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#OpenSource</a> <a href="https://qoto.org/tags/Networking" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Networking</a>

Alterego_Midshipman**A reference server project for researching the possibilities of moving YaCy to an alternative codebase**<a href="https://legendllc.blogspot.com/2025/01/a-reference-server-project-for.html" rel="nofollow noopener noreferrer" target="_blank">https://legendllc.blogspot.com/2025/01/a-reference-server-project-for.html</a>**Introduction** YaCy is an open source decentralized search engine written in Java. However, its performance and scalability are limited by the current selection of technologies. This project involves the creation of a reference server for testing the transition of YaCy to more efficient programming languages, such as C++, C, Rust and Go. **Goal** **Productivity and efficiency research** alternative languages in the development of a decentralized search system. **Optimization of the use of hardware resources**, including multiprocessor systems, large amounts of RAM and GRAID. **Providing better multithreading support** and expansion of search algorithm capabilities. **Reduced dependency on the JVM** to increase performance and reduce resource usage. **Hardware platform** The project involves the use of a reference server with the following characteristics: **RAM**: 1–10 TB (depending on configuration and indexing volume) **Processors**: 4-16 Intel server processors **Co-processors**: Nvidia graphics cards for processing large amounts of data **Refuge**: GRAID to increase data access speed and reduce latency **Network interaction**: Optimized network protocols for efficient exchange of information between nodes **The main stages of development** Analysis of the current YaCy architecture and identification of key limitations. Selection of the appropriate programming language (C++, C, Rust, Go) and testing of their capabilities. Development of a prototype of an alternative core of the search engine. Integration with existing infrastructure and performance testing on a reference server. Evaluation of efficiency and further improvement of algorithms. **Expected results** Improved performance of the search engine due to the efficient use of multi-core processors and GRAID. Reduced RAM usage and increased indexing speed. Ensuring greater stability and scalability of the system. Possibility of further adaptation of the new code base for use in other decentralized projects. This project will become the basis for future improvements in the field of decentralized search systems and will demonstrate the possibility of creating a high-performance alternative to YaCy using modern technologies. A reference server project for deploying and exploring the possibilities of transitioning YaCy to the C++, C, Rust, Go code base with 1-10 TB RAM and GRAID, with 4-16 Intel processors and NVidia cards as co-processors .This is a very ambitious and exciting project! Here are some suggestions for server configuration and budgeting for such a study. **Reference server configuration:** **RAM**:**1–10 TB**: Large amount of memory allows efficient processing and indexing of large amounts of data while maintaining low query latency. It is recommended to use servers with scalable memory support, e.g. **DDR4 ECC RAM** for stability. **Processors (CPU)**:**4-16 Intel server processors**: Multi-core processors, e.g. **Intel Xeon** or **Intel Cascade Lake**, which support large computing loads and high throughput. It is suggested to consider **multi-processor systems**, which allow parallel computations to be handled and improve overall performance. **Graphics cards (GPU)**:**Nvidia GPU** to process large amounts of data, including usage **DIFFERENT** to speed up certain processes, such as data analysis or deep learning. Type cards are suitable for this **Nvidia Tesla V100** or **A100**, which are specifically focused on large-scale computing and data processing. **Storage (Storage)**:**GRADE** — technology for increasing storage performance. Using **NVMe SSD** in combination with **RAID arrays** helps minimize delays when accessing data. **Terabytes of storage** for indexing large volumes of web data. **Networking**:**Optimized network protocols**: use **10G Ethernet** or **InfiniBand** for fast data transfer between nodes in the network and to support scalability. **Technologies for development:** **Programming languages**:**C++/C**: for the most efficient use of hardware resources, high performance and the possibility of manual memory management. **Rust**: due to its security and high efficiency, it will be an excellent choice for systems working with rich flows. **Go**: convenient for parallel computing and for building scalable network applications, which is well suited for decentralized systems. Comparing the performance of these languages will allow to determine the optimal one for further stages of development. **Search and indexing algorithms**:Determination of effective algorithms for processing large volumes of data. Development of algorithms for optimization of multi-threading and work with multi-core processors. **Estimate:** **Equipment**:**Intel Xeon servers with 4-16 processors**: around $10,000-$50,000 depending on the number of processors and the level of computing power. **Nvidia Tesla GPU**: approximately $5,000-$15,000 per card. **RAM**: 1 TB RAM — $10,000-$30,000. **Refuge** (NVMe SSD/GRAID): around $2,000-$10,000 for high performance solutions. **Licenses and Software**:**Operating system**: Free based solutions **Linux** (e.g. Ubuntu). **Development tools**: depending on the programming language, may be free (for example, for Rust, Go). **Infrastructure and testing**:Using cloud-based testing platforms can add about $1,000-$5,000 per year to the budget for additional capacity and monitoring tools. **Expected results:** **Improved performance** thanks to more efficient use of multi-core processors and high-performance GPUs. **Reducing delays and resource utilization** due to optimization of indexing and query processing. **Scalability** systems for processing big data and search queries. **Open source and adaptation** for other decentralized projects that can contribute to the development of technologies in the field of search engines. This is definitely a big project that can significantly affect the evolution of decentralized search engines. Here is a table that shows the maximum configuration of the reference server and a rough estimate for researching the transition of YaCy to alternative codebases (C++, C, Rust, Go): **Component** **Maximum configuration** **Estimated cost** **Processors (CPU)** 16 Intel Xeon server processors (Cascade Lake) 40,000–80,000 dollars **RAM** 10 TB DDR4 ECC RAM $100,000–$300,000 **Graphics cards (GPU)** 4 Nvidia Tesla V100 or A100 (for data processing and acceleration) 20,000–60,000 dollars **Storage (Storage)** 100 TB NVMe SSD + GRAID for high performance storage 20,000–40,000 dollars **Network protocols** 10G Ethernet or InfiniBand for fast data transfer $5,000–$15,000 **Operating system** Ubuntu or other free Linux distributions Free **Development and testing tools** Docker, Kubernetes, profiling and monitoring tools $1,000-$5,000 (Licenses, Tools) **Power consumption and cooling** Energy consumption for such systems including cooling $5,000-$10,000 per year **General estimate (approximate)** **$190,000 to $510,000** **Notes:** **Processors**: The cost of Intel Xeon (Cascade Lake) server processors may vary by number of cores and speed, as well as by specific models. **RAM**: The cost of memory for servers with large amounts of memory (up to 10 TB) increases significantly, so the price may vary depending on the amount. **Graphics cards (GPU)**: The cost of Nvidia Tesla depends on specific models. For the tasks of processing large volumes of data and acceleration, you can use models of the V100 or A100 type. **Refuge**: This amount of storage will require usage **NVMe SSD** in combination with **GRADE**, which will significantly improve the speed of data access. **Network protocols**: For high-speed data transmission between nodes, it is important to have a network infrastructure of the type **10G Ethernet** or **InfiniBand**. This is an approximate estimate for the maximum configuration, based on typical server components for such projects. These costs may vary depending on specific hardware vendors and licenses. In order to implement a reference server to study the transition of YaCy to alternative codebases, it is important to develop a clear budget for the implementer, as well as a list of potential candidates who can be involved in the project. **Estimate of the implementer:** **Stage** **Description** **Estimated cost** **Design and planning** Development of the technical task, preparation of the project implementation plan 10,000–20,000 dollars **Selection and purchase of equipment** Evaluation and purchase of server equipment, network components, GPU 200,000–500,000 dollars **Server settings** Installation and configuration of servers, including network and data storage settings 30,000–50,000 dollars **Development of a system core prototype** Development of an initial prototype of an alternative YaCy kernel in new languages 40,000–80,000 dollars **Integration with existing infrastructure** Integrating the new code base with the current YaCy infrastructure 20,000–40,000 dollars **Testing and monitoring** Performance, stability and scalability tests 20,000–40,000 dollars **Evaluation of results and optimization** Evaluation of efficiency, introduction of optimizations and corrections of algorithms 10,000–20,000 dollars **Documentation and training** Preparation of documentation, staff training and project support 10,000–15,000 dollars **Total estimated cost** **$340,000 to $765,000** **List of candidates for implementation:** **Technical project manager (Project Manager)** Responsible for overall project coordination, planning and organization of works. Requirements: experience in IT project management, knowledge of Agile or Waterfall methodologies, experience in managing large teams. **System architect** Develops the overall architecture of the server infrastructure, is responsible for optimization and scalability. Requirements: experience working with highly loaded systems, knowledge of C++, Rust, Go technologies, experience in building and configuring server solutions. **Developers (C++, C, Rust, Go)** A new search engine core is being developed in selected programming languages. Requirements: in-depth knowledge of relevant programming languages, experience in developing scalable applications, familiarity with decentralized systems and search algorithms. **Server configuration engineer** Responsible for physical setup and configuration of servers, including GRAID deployment, configuration of graphics cards and network components. Requirements: experience in configuring server systems and working with highly loaded infrastructures. **Test Engineer (QA)** Designs and runs tests to verify system performance, scalability, and reliability. Requirements: experience in testing distributed and highly loaded systems, knowledge of testing tools such as JMeter or other similar tools. **Monitoring and optimization engineer** Responsible for monitoring system performance, finding and eliminating performance bottlenecks. Requirements: experience in setting up system monitoring and profiling systems (Prometheus, Grafana, or similar), optimization of resource use. **Documentation and technical writer** Responsible for the preparation of technical documentation, description of algorithms, instructions for setting up and operating the system. Requirements: experience in writing technical documentation for complex IT projects. **Total costs for the implementer:** **Team salary**:Technical Manager: $80,000–120,000 per year System Architect: $90,000-$150,000 per year Developers: $70,000-$120,000 per year (each) Configuration and Test Engineers: $60,000-$100,000 per year (each) Documentation and technical writer: $50,000-$80,000 per year **Implementation time**: approximately 12–18 months (depending on the complexity and scope of the project).**License and tool costs**:Development and testing software: $5,000-$10,000 Licenses for server operating systems and monitoring tools: $10,000–20,000 This estimate is indicative and may vary depending on specific circumstances, suppliers and selected solutions for project implementation. When considering taxes in the EU, particularly in the Netherlands, it is necessary to include corporate income taxes, VAT (value added tax) and other possible financing costs. The Netherlands has one of the most transparent and efficient tax systems in the EU, but it is important to consider various aspects. **Taxes in the Netherlands for IT projects:** **Corporate Income Tax (CIT):** The income tax rate in the Netherlands is 19% for income up to €200,000 and 25.8% for income above this threshold. **VAT:** The standard VAT rate in the Netherlands is 21%. For certain types of goods and services, reduced rates are possible (for example, 9% for some goods and services). **Social contributions:** Salary expenses are subject to social contributions. This includes pension and medical contributions, which are approximately 27.65% of workers' wages. **Taxes on dividends:** A dividend tax of 15% is applied to the payment of dividends at the enterprise level. **Other fees:** The Netherlands has some additional fees for certain activities, including contributions to environmental initiatives or taxes on the use of specific resources. **Accounting for taxes in the project estimate:** **Stage** **Estimated cost before taxes** **Tax expenses (approx.)** **Approximate cost with taxes** **Design and planning** 10,000–20,000 dollars $2,000-$4,000 (20%) $12,000–$24,000 **Selection and purchase of equipment** 200,000–500,000 dollars $42,000-$105,000 (21%) $242,000 to $605,000 **Server settings** 30,000–50,000 dollars $6,300-$10,500 (21%) $36,300–$60,500 **Development of a system core prototype** 40,000–80,000 dollars $8,400-$16,800 (21%) $48,400–$96,800 **Integration with infrastructure** 20,000–40,000 dollars $4,200-$8,400 (21%) $24,200–$48,400 **Testing and monitoring** 20,000–40,000 dollars $4,200-$8,400 (21%) $24,200–$48,400 **Evaluation of results and optimization** 10,000–20,000 dollars $2,000-$4,000 (20%) $12,000–$24,000 **Documentation and training** 10,000–15,000 dollars $2,100-$3,150 (21%) $12,100-$18,150 **The total cost of the project** $340,000 to $765,000 $71,100-$161,850 $411,100 to $926,850 **Additional costs for social contributions:** If the company hires employees, social security contributions can further increase wage costs (approximately 27.65%). This applies to the hiring of both local specialists and foreign specialists if they work within the Netherlands. **Estimated total cost with taxes for the project:** Taking into account income taxes, VAT and other costs, the total cost of the project based on the maximum targeted costs will look like this: **Approximate cost without taxes:** $340,000 – $765,000 **Estimated tax costs (approximately 21%):** 71,100 - 161,850 dollars **Approximate cost with taxes:** 411,100 - 926,850 dollars These costs may vary depending on various factors, such as the exact VAT rates, the specifics of social contributions and possible changes in Dutch tax law. For project management and cost estimation, it is important to choose companies that specialize in technical consulting, large IT project management, and have experience in areas such as decentralized systems development, big data processing, and infrastructure scaling. Here are several categories of companies and a list of candidates that may be involved in such a project: **Types of companies for project management:** **IT consulting and system integration:** They specialize in the development, testing and integration of complex IT systems. They have experience in managing projects with large technical requirements and preparing relevant estimates. **Infrastructure technology consulting companies:** Support project teams in the development of scalable and high-performance solutions that require significant resources. **Software development outsourcing company:** Connecting specialists to specific tasks of developing the search engine core and optimizing performance. **Possible candidates for project management:** **Accenture** **Description:** Global consulting company with vast experience in the field of IT consulting, development and project management at the international level. **Field of specialization:** Development of large corporate solutions, optimization and scaling of infrastructure. **Why choose:** Has experience in working with large technological projects, such as large search engines and data processing. **Get hold of it** **Description:** A large international consulting company with strong positions in technology project management, including automation and digital transformation. **Field of specialization:** Software development, introduction of new technologies (including C++, Rust, Go). **Why choose:** He has experience in implementing technologies for large distribution and high-performance systems. **IBM Global Services** **Description:** A classic provider of services for the development and scaling of IT systems. **Field of specialization:** Software development, consulting on infrastructure optimization, including for processing large volumes of data. **Why choose:** He has extensive experience in infrastructure projects and the deployment of complex technologies. **Tata Consultancy Services (TCS)** **Description:** One of the leaders in the field of IT consulting, known for its powerful resources in the field of software development and infrastructure. **Field of specialization:** Consulting, software development, management of large projects, infrastructure support. **Why choose:** Extensive experience in creating and optimizing distribution and large systems. **Wipro** **Description:** An Indian outsourcing company with a global scale and experience in working with large projects in various industries. **Field of specialization:** Software development, testing, as well as strategic project management based on modern technologies. **Why choose:** Specializes in large IT projects, including the integration of various technologies for distribution systems. **Deloitte Consulting** **Description:** One of the largest consulting companies in the world, providing services in the field of IT consulting, finance, management consulting and strategic planning. **Field of specialization:** Project management, development of infrastructure and scalable solutions, optimization of resources. **Why choose:** He has experience in the implementation of technologies for large infrastructure and technological projects. **List of candidates for project management positions:** **Technical Director (CTO):** **Candidates:** Senior Solution Architect with experience in developing scalable distributed systems. Experienced engineer in optimizing productivity in working with big data. **Project Manager (PM):** **Candidates:** Experienced in managing large IT software development projects, including infrastructure and scaling. **Software Architect:** **Candidates:** Specialists with experience in C++, Rust, Go, Java for the development of search engines. **Big Data and Infrastructure Engineers:** **Candidates:** Engineers with experience in GRAID, distributed systems and big data processing. **Testers and QA specialists:** **Candidates:** Specialists in testing high-performance systems for big data. **Cost of project manager services:** **Company** **Approximate cost of services (monthly)** **Note** **Accenture** 50,000–150,000 euros High level of service for large projects with large-scale infrastructures. **Get hold of it** 40,000–120,000 euros Offers services for development and optimization of complex systems. **IBM Global Services** 60,000–200,000 euros Support in the development and integration of complex technologies. **Wipro** 30,000–100,000 euros Provision of complex software and infrastructure development services. **Deloitte** 70,000–180,000 euros Offers consulting for large transformational IT projects. These cost estimates may vary depending on the complexity of the project, specialization and region in which the company is located.Keywords: YaCy, P2P, search engine, C++, Rust, Go, GRAID, Intel, NVidia, co-processors, big computing, decentralized search. <a href="https://qoto.org/tags/YaCy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#YaCy</a> <a href="https://qoto.org/tags/P2P" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#P2P</a> <a href="https://qoto.org/tags/SearchEngine" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#SearchEngine</a> <a href="https://qoto.org/tags/Cplusplus" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Cplusplus</a> <a href="https://qoto.org/tags/Rust" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Rust</a> <a href="https://qoto.org/tags/GoLang" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GoLang</a> <a href="https://qoto.org/tags/GRAID" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#GRAID</a> <a href="https://qoto.org/tags/Intel" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Intel</a> <a href="https://qoto.org/tags/Nvidia" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Nvidia</a> <a href="https://qoto.org/tags/Decentralized" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Decentralized</a> <a href="https://qoto.org/tags/BigData" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#BigData</a> <a href="https://qoto.org/tags/AI" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#AI</a> <a href="https://qoto.org/tags/OpenSource" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#OpenSource</a> <a href="https://qoto.org/tags/Networking" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Networking</a>

Robert WinklerI just moved from <a href="https://mstdn.social/tags/Medium" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Medium</a> to a <a href="https://mstdn.social/tags/selfhosted" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#selfhosted</a> <a href="https://mstdn.social/tags/Fediverse" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Fediverse</a> <a href="https://mstdn.social/tags/Blog" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Blog</a> <a href="https://writefreely.nube-gran.de/" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@robert-winkler</a>, using <a href="https://writing.exchange/@writefreely" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@writefreely</a>. My first post is about <a href="https://mstdn.social/tags/tracking" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#tracking</a> -free <a href="https://mstdn.social/tags/search" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#search</a> engines and <a href="https://mstdn.social/tags/web" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#web</a> crawling. I'm hosting two search engines: <a href="https://mstdn.social/tags/LibreY" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#LibreY</a> and <a href="https://mstdn.social/tags/YaCy" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#YaCy</a>. I really like the simplistic layout and the <a href="https://mstdn.social/tags/Fediverse" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#Fediverse</a> integration: <a href="https://writefreely.nube-gran.de/self-hosted-web-searches-with-librey-and-yacy" rel="nofollow noopener noreferrer" translate="no" target="_blank">https://writefreely.nube-gran.de/self-hosted-web-searches-with-librey-and-yacy</a>. <a href="https://mstdn.social/tags/toxicTech" class="mention hashtag" rel="nofollow noopener noreferrer" target="_blank">#toxicTech</a> <a href="https://hachyderm.io/@cyberlyra" class="u-url mention" rel="nofollow noopener noreferrer" target="_blank">@cyberlyra</a>

Recent searches

Search options

Administered by:

Server stats:

#yacy