Compare commits
2 Commits
master
...
docker-1.0
| Author | SHA1 | Date | |
|---|---|---|---|
| 83c8a432ce | |||
| 8452830cf0 |
4
.gitignore
vendored
4
.gitignore
vendored
@ -2,7 +2,7 @@
|
||||
**/.settings
|
||||
**/build
|
||||
.gradle
|
||||
**/.project
|
||||
**/.classpath
|
||||
.project
|
||||
.classpath
|
||||
**/*.rej
|
||||
**/*.orig
|
||||
|
||||
@ -4,7 +4,7 @@ FROM jlesage/baseimage-gui:alpine-3.10-glibc
|
||||
ARG DOCKER_IMAGE_VERSION=unknown
|
||||
|
||||
# JDK version
|
||||
ARG JDK=11
|
||||
ARG JDK=9
|
||||
|
||||
# Important directories
|
||||
ARG TMP_DIR=/muwire-tmp
|
||||
|
||||
19
README.md
19
README.md
@ -1,10 +1,8 @@
|
||||
The GitHub repo is mirrored from the in-I2P GitLab repo. Please open PRs and issues at http://git.idk.i2p/zlatinb/muwire
|
||||
|
||||
# MuWire - Easy Anonymous File-Sharing
|
||||
|
||||
MuWire is an easy to use file-sharing program which offers anonymity using [I2P technology](http://geti2p.net). It works on any platform Java works on, including Windows,MacOS,Linux.
|
||||
|
||||
The current stable release - 0.7.4 is avaiable for download at https://muwire.com. The latest plugin build and instructions how to install the plugin are available inside I2P at http://muwire.i2p.
|
||||
The current stable release - 0.6.8 is avaiable for download at https://muwire.com. The latest plugin build and instructions how to install the plugin are available inside I2P at http://muwire.i2p.
|
||||
|
||||
You can find technical documentation in the [doc] folder. Also check out the [Wiki] for various other documentation.
|
||||
|
||||
@ -21,7 +19,7 @@ If you want to run the unit tests, type
|
||||
./gradlew clean build
|
||||
```
|
||||
|
||||
If you want to build binary bundles that do not depend on Java or I2P, see the [muwire-pkg] project. If you want to package MuWire for a Linux distribution, see the [Packaging] wiki page.
|
||||
If you want to build binary bundles that do not depend on Java or I2P, see the [muwire-pkg] project
|
||||
|
||||
## Running the GUI
|
||||
|
||||
@ -30,7 +28,9 @@ Type
|
||||
./gradlew gui:run
|
||||
```
|
||||
|
||||
The setup wizard will ask you for the host and port of an I2P or I2Pd router.
|
||||
If you have an I2P router running on the same machine that is all you need to do. If you use a custom I2CP host and port, create a file `i2p.properties` and put `i2cp.tcp.host=<host>` and `i2cp.tcp.port=<port>` in there. On Windows that file should go into `%HOME%\AppData\Roaming\MuWire`, on Mac into `$HOME/Library/Application Support/MuWire` and on Linux `$HOME/.MuWire`
|
||||
|
||||
[Default I2CP port]\: `7654`
|
||||
|
||||
## Running the CLI
|
||||
|
||||
@ -49,12 +49,6 @@ MuWire is available as a Docker image. For more information see the [Docker] pa
|
||||
## Translations
|
||||
If you want to help translate MuWire, instructions are on the wiki https://github.com/zlatinb/muwire/wiki/Translate
|
||||
|
||||
## Related Projects
|
||||
### MuWire Tracker Daemon
|
||||
The MuWire Tracker Daemon (or mwtrackerd for short) is a project to bring functionality similar to BitTorrent tracking to MuWire. For more info see the [Tracker] page.
|
||||
### MuCats
|
||||
MuCats is a project to create a website for hosting hashes of files shared on the MuWire network. For more info see the [MuCats] project.
|
||||
|
||||
## GPG Fingerprint
|
||||
|
||||
```
|
||||
@ -68,11 +62,8 @@ You can find the full key at https://keybase.io/zlatinb
|
||||
[Wiki]: https://github.com/zlatinb/muwire/wiki
|
||||
[doc]: https://github.com/zlatinb/muwire/tree/master/doc
|
||||
[muwire-pkg]: https://github.com/zlatinb/muwire-pkg
|
||||
[Packaging]: https://github.com/zlatinb/muwire/wiki/Packaging
|
||||
[cli options]: https://github.com/zlatinb/muwire/wiki/CLI-Configuration-Options
|
||||
[I2P Github]: https://github.com/i2p/i2p.i2p
|
||||
[Plugin]: https://github.com/zlatinb/muwire/wiki/Plugin
|
||||
[Docker]: https://github.com/zlatinb/muwire/wiki/Docker
|
||||
[jlesage/docker-baseimage-gui]: https://github.com/jlesage/docker-baseimage-gui
|
||||
[Tracker]: https://github.com/zlatinb/muwire/wiki/Tracker-Daemon
|
||||
[MuCats]: https://github.com/zlatinb/mucats
|
||||
|
||||
15
TODO.md
15
TODO.md
@ -15,14 +15,16 @@ This helps with scalability
|
||||
* Metadata parsing and search
|
||||
* Automatic adjustment of number of I2P tunnels
|
||||
* Persist trust immediately
|
||||
* Check if user-selected download and incomplete locations exist and are writeable
|
||||
* Enum i18n
|
||||
* Ability to share trust list only with trusted users
|
||||
* Confidential files visible only to certain users
|
||||
* Download queue with priorities
|
||||
* Use tracker pings - either embedded logic or external mwtrackerd to add more sources to downloads
|
||||
* Public Feed feature
|
||||
|
||||
### Chat
|
||||
* echo "unknown/innappropriate command" in the console
|
||||
* break up lines on CR/LF, send multiple messages
|
||||
* Style timestamps and persona names
|
||||
* enforce # in room names or ignore it
|
||||
* auto-create/join channel on server start
|
||||
* jump from notification window to room with message
|
||||
@ -30,16 +32,11 @@ This helps with scalability
|
||||
### Swing GUI
|
||||
* I2P Status panel - display message when connected to external router
|
||||
* Search box - left identation
|
||||
* Ability to disable switching of tabs on actions
|
||||
|
||||
### Web UI/Plugin
|
||||
* HTML 5 media players
|
||||
* Minimal dependency (break up groovy-all.jar)
|
||||
* Remove versions from jar names
|
||||
* Security: POST nonces, CSP headers
|
||||
* Upload files from browser to plugin via drag-and-drop
|
||||
* Check permissions, display better errors when sharing local folders
|
||||
|
||||
### mwtrackerd
|
||||
* `save` and `load` JSON-RPC commands that save and load swarm state respectively
|
||||
* load-test with many many hashes (1M?)
|
||||
* evaluate other usage scenarios besides website backend
|
||||
|
||||
|
||||
21
build.gradle
21
build.gradle
@ -2,26 +2,13 @@ subprojects {
|
||||
apply plugin: 'groovy'
|
||||
|
||||
dependencies {
|
||||
compile "org.codehaus.groovy:groovy:${groovyVersion}"
|
||||
compile "org.codehaus.groovy:groovy-jsr223:${groovyVersion}"
|
||||
compile "org.codehaus.groovy:groovy-json:${groovyVersion}"
|
||||
compile 'org.codehaus.groovy:groovy:2.4.15'
|
||||
compile 'org.codehaus.groovy:groovy-jsr223:2.4.15'
|
||||
compile 'org.codehaus.groovy:groovy-json:2.4.15'
|
||||
}
|
||||
|
||||
compileGroovy {
|
||||
groovyOptions.optimizationOptions.indy = false
|
||||
sourceCompatibility = project.sourceCompatibility
|
||||
targetCompatibility = project.targetCompatibility
|
||||
options.compilerArgs += project.compilerArgs
|
||||
options.deprecation = true
|
||||
options.encoding = 'UTF-8'
|
||||
}
|
||||
|
||||
compileJava {
|
||||
sourceCompatibility = project.sourceCompatibility
|
||||
targetCompatibility = project.targetCompatibility
|
||||
options.compilerArgs += project.compilerArgs
|
||||
options.deprecation = true
|
||||
options.encoding = 'UTF-8'
|
||||
groovyOptions.optimizationOptions.indy = true
|
||||
}
|
||||
|
||||
repositories {
|
||||
|
||||
@ -32,7 +32,7 @@ import com.muwire.core.UILoadedEvent
|
||||
import com.muwire.core.files.AllFilesLoadedEvent
|
||||
|
||||
class CliLanterna {
|
||||
private static final String MW_VERSION = "0.7.4"
|
||||
private static final String MW_VERSION = "0.6.8"
|
||||
|
||||
private static volatile Core core
|
||||
|
||||
|
||||
@ -3,7 +3,6 @@ package com.muwire.clilanterna
|
||||
import com.googlecode.lanterna.gui2.TextGUIThread
|
||||
import com.googlecode.lanterna.gui2.table.TableModel
|
||||
import com.muwire.core.Core
|
||||
import com.muwire.core.InfoHash
|
||||
import com.muwire.core.SharedFile
|
||||
import com.muwire.core.files.AllFilesLoadedEvent
|
||||
import com.muwire.core.files.DirectoryWatchedEvent
|
||||
@ -28,6 +27,7 @@ class FilesModel {
|
||||
core.eventBus.register(FileLoadedEvent.class, this)
|
||||
core.eventBus.register(FileUnsharedEvent.class, this)
|
||||
core.eventBus.register(FileHashedEvent.class, this)
|
||||
core.eventBus.register(AllFilesLoadedEvent.class, this)
|
||||
|
||||
Runnable refreshModel = {refreshModel()}
|
||||
Timer timer = new Timer(true)
|
||||
@ -37,6 +37,15 @@ class FilesModel {
|
||||
|
||||
}
|
||||
|
||||
void onAllFilesLoadedEvent(AllFilesLoadedEvent e) {
|
||||
def eventBus = core.eventBus
|
||||
guiThread.invokeLater {
|
||||
core.muOptions.watchedDirectories.each {
|
||||
eventBus.publish(new FileSharedEvent(file: new File(it)))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void onFileLoadedEvent(FileLoadedEvent e) {
|
||||
guiThread.invokeLater {
|
||||
sharedFiles.add(e.loadedFile)
|
||||
@ -63,7 +72,7 @@ class FilesModel {
|
||||
sharedFiles.each {
|
||||
long size = it.getCachedLength()
|
||||
boolean comment = it.comment != null
|
||||
boolean certified = core.certificateManager.hasLocalCertificate(new InfoHash(it.getRoot()))
|
||||
boolean certified = core.certificateManager.hasLocalCertificate(it.getInfoHash())
|
||||
String hits = String.valueOf(it.getHits())
|
||||
String downloaders = String.valueOf(it.getDownloaders().size())
|
||||
model.addRow(new SharedFileWrapper(it), DataHelper.formatSize2(size, false)+"B", comment, certified, hits, downloaders)
|
||||
|
||||
@ -21,6 +21,7 @@ import com.muwire.core.filecert.UICreateCertificateEvent
|
||||
import com.muwire.core.files.DirectoryUnsharedEvent
|
||||
import com.muwire.core.files.FileSharedEvent
|
||||
import com.muwire.core.files.FileUnsharedEvent
|
||||
import com.muwire.core.files.UIPersistFilesEvent
|
||||
|
||||
class FilesView extends BasicWindow {
|
||||
private final FilesModel model
|
||||
@ -83,6 +84,7 @@ class FilesView extends BasicWindow {
|
||||
|
||||
Button unshareButton = new Button("Unshare", {
|
||||
core.eventBus.publish(new FileUnsharedEvent(unsharedFile : sf))
|
||||
core.eventBus.publish(new UIPersistFilesEvent())
|
||||
MessageDialog.showMessageDialog(textGUI, "File Unshared", "Unshared "+sf.getFile().getName(), MessageDialogButton.OK)
|
||||
} )
|
||||
Button addCommentButton = new Button("Add Comment", {
|
||||
|
||||
@ -1,42 +1,12 @@
|
||||
|
||||
plugins {
|
||||
id 'java-library'
|
||||
id 'maven-publish'
|
||||
}
|
||||
|
||||
apply plugin : 'application'
|
||||
mainClassName = 'com.muwire.core.Core'
|
||||
applicationDefaultJvmArgs = ['-Djava.util.logging.config.file=logging.properties']
|
||||
dependencies {
|
||||
api "net.i2p:i2p:${i2pVersion}"
|
||||
api "net.i2p:router:${i2pVersion}"
|
||||
api "net.i2p.client:mstreaming:${i2pVersion}"
|
||||
implementation "net.i2p.client:streaming:${i2pVersion}"
|
||||
compile "net.i2p:i2p:${i2pVersion}"
|
||||
compile "net.i2p:router:${i2pVersion}"
|
||||
compile "net.i2p.client:mstreaming:${i2pVersion}"
|
||||
compile "net.i2p.client:streaming:${i2pVersion}"
|
||||
|
||||
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.4.2'
|
||||
testImplementation 'junit:junit:4.12'
|
||||
testImplementation 'org.codehaus.groovy:groovy-all:3.0.4'
|
||||
}
|
||||
|
||||
|
||||
// this is necessary because applying both groovy and java-library doesn't work well
|
||||
configurations {
|
||||
apiElements.outgoing.variants {
|
||||
classes {
|
||||
artifact file: compileGroovy.destinationDir, builtBy: compileGroovy
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
configurations.testImplementation {
|
||||
exclude group:'org.codehaus.groovy', module:'groovy-testng'
|
||||
}
|
||||
|
||||
// publish core to local maven repo for sister projects
|
||||
publishing {
|
||||
publications {
|
||||
muCore(MavenPublication) {
|
||||
from components.java
|
||||
}
|
||||
}
|
||||
repositories {
|
||||
mavenLocal()
|
||||
}
|
||||
testCompile 'org.junit.jupiter:junit-jupiter-api:5.4.2'
|
||||
testCompile 'junit:junit:4.12'
|
||||
}
|
||||
|
||||
@ -1,12 +1,7 @@
|
||||
package com.muwire.core
|
||||
|
||||
import com.muwire.core.files.PersisterDoneEvent
|
||||
import com.muwire.core.files.PersisterFolderService
|
||||
|
||||
import java.nio.charset.StandardCharsets
|
||||
import java.util.concurrent.atomic.AtomicBoolean
|
||||
import java.util.logging.Level
|
||||
import java.util.zip.ZipException
|
||||
|
||||
import com.muwire.core.chat.ChatDisconnectionEvent
|
||||
import com.muwire.core.chat.ChatManager
|
||||
@ -23,10 +18,8 @@ import com.muwire.core.connection.I2PAcceptor
|
||||
import com.muwire.core.connection.I2PConnector
|
||||
import com.muwire.core.connection.LeafConnectionManager
|
||||
import com.muwire.core.connection.UltrapeerConnectionManager
|
||||
import com.muwire.core.download.DownloadHopelessEvent
|
||||
import com.muwire.core.download.DownloadManager
|
||||
import com.muwire.core.download.SourceDiscoveredEvent
|
||||
import com.muwire.core.download.SourceVerifiedEvent
|
||||
import com.muwire.core.download.UIDownloadCancelledEvent
|
||||
import com.muwire.core.download.UIDownloadEvent
|
||||
import com.muwire.core.download.UIDownloadPausedEvent
|
||||
@ -36,18 +29,9 @@ import com.muwire.core.filecert.CertificateManager
|
||||
import com.muwire.core.filecert.UICreateCertificateEvent
|
||||
import com.muwire.core.filecert.UIFetchCertificatesEvent
|
||||
import com.muwire.core.filecert.UIImportCertificateEvent
|
||||
import com.muwire.core.filefeeds.FeedClient
|
||||
import com.muwire.core.filefeeds.FeedFetchEvent
|
||||
import com.muwire.core.filefeeds.FeedItemFetchedEvent
|
||||
import com.muwire.core.filefeeds.FeedManager
|
||||
import com.muwire.core.filefeeds.UIDownloadFeedItemEvent
|
||||
import com.muwire.core.filefeeds.UIFilePublishedEvent
|
||||
import com.muwire.core.filefeeds.UIFeedConfigurationEvent
|
||||
import com.muwire.core.filefeeds.UIFeedDeletedEvent
|
||||
import com.muwire.core.filefeeds.UIFeedUpdateEvent
|
||||
import com.muwire.core.filefeeds.UIFileUnpublishedEvent
|
||||
import com.muwire.core.files.FileDownloadedEvent
|
||||
import com.muwire.core.files.FileHashedEvent
|
||||
import com.muwire.core.files.FileHashingEvent
|
||||
import com.muwire.core.files.FileHasher
|
||||
import com.muwire.core.files.FileLoadedEvent
|
||||
import com.muwire.core.files.FileManager
|
||||
@ -57,11 +41,7 @@ import com.muwire.core.files.HasherService
|
||||
import com.muwire.core.files.PersisterService
|
||||
import com.muwire.core.files.SideCarFileEvent
|
||||
import com.muwire.core.files.UICommentEvent
|
||||
import com.muwire.core.files.directories.UISyncDirectoryEvent
|
||||
import com.muwire.core.files.directories.WatchedDirectoryConfigurationEvent
|
||||
import com.muwire.core.files.directories.WatchedDirectoryConvertedEvent
|
||||
import com.muwire.core.files.directories.WatchedDirectoryConverter
|
||||
import com.muwire.core.files.directories.WatchedDirectoryManager
|
||||
import com.muwire.core.files.UIPersistFilesEvent
|
||||
import com.muwire.core.files.AllFilesLoadedEvent
|
||||
import com.muwire.core.files.DirectoryUnsharedEvent
|
||||
import com.muwire.core.files.DirectoryWatchedEvent
|
||||
@ -72,7 +52,6 @@ import com.muwire.core.hostcache.HostDiscoveredEvent
|
||||
import com.muwire.core.mesh.MeshManager
|
||||
import com.muwire.core.search.BrowseManager
|
||||
import com.muwire.core.search.QueryEvent
|
||||
import com.muwire.core.search.ResponderCache
|
||||
import com.muwire.core.search.ResultsEvent
|
||||
import com.muwire.core.search.ResultsSender
|
||||
import com.muwire.core.search.SearchEvent
|
||||
@ -88,7 +67,6 @@ import com.muwire.core.upload.UploadManager
|
||||
import com.muwire.core.util.MuWireLogManager
|
||||
import com.muwire.core.content.ContentControlEvent
|
||||
import com.muwire.core.content.ContentManager
|
||||
import com.muwire.core.tracker.TrackerResponder
|
||||
|
||||
import groovy.util.logging.Log
|
||||
import net.i2p.I2PAppContext
|
||||
@ -96,8 +74,10 @@ import net.i2p.client.I2PClientFactory
|
||||
import net.i2p.client.I2PSession
|
||||
import net.i2p.client.streaming.I2PSocketManager
|
||||
import net.i2p.client.streaming.I2PSocketManagerFactory
|
||||
import net.i2p.client.streaming.I2PSocketOptions
|
||||
import net.i2p.client.streaming.I2PSocketManager.DisconnectListener
|
||||
import net.i2p.crypto.DSAEngine
|
||||
import net.i2p.crypto.SigType
|
||||
import net.i2p.data.Destination
|
||||
import net.i2p.data.PrivateKey
|
||||
import net.i2p.data.Signature
|
||||
@ -116,20 +96,18 @@ public class Core {
|
||||
final Properties i2pOptions
|
||||
final MuWireSettings muOptions
|
||||
|
||||
final I2PSession i2pSession;
|
||||
private I2PSocketManager i2pSocketManager
|
||||
private final I2PSession i2pSession;
|
||||
final TrustService trustService
|
||||
final TrustSubscriber trustSubscriber
|
||||
private final PersisterService persisterService
|
||||
private final PersisterFolderService persisterFolderService
|
||||
final HostCache hostCache
|
||||
final ConnectionManager connectionManager
|
||||
private final HostCache hostCache
|
||||
private final ConnectionManager connectionManager
|
||||
private final CacheClient cacheClient
|
||||
private final UpdateClient updateClient
|
||||
final ConnectionAcceptor connectionAcceptor
|
||||
private final ConnectionAcceptor connectionAcceptor
|
||||
private final ConnectionEstablisher connectionEstablisher
|
||||
private final HasherService hasherService
|
||||
final DownloadManager downloadManager
|
||||
private final DownloadManager downloadManager
|
||||
private final DirectoryWatcher directoryWatcher
|
||||
final FileManager fileManager
|
||||
final UploadManager uploadManager
|
||||
@ -137,11 +115,6 @@ public class Core {
|
||||
final CertificateManager certificateManager
|
||||
final ChatServer chatServer
|
||||
final ChatManager chatManager
|
||||
final FeedManager feedManager
|
||||
private final FeedClient feedClient
|
||||
private final WatchedDirectoryConverter watchedDirectoryConverter
|
||||
final WatchedDirectoryManager watchedDirectoryManager
|
||||
private final TrackerResponder trackerResponder
|
||||
|
||||
private final Router router
|
||||
|
||||
@ -158,11 +131,7 @@ public class Core {
|
||||
// Read defaults
|
||||
def defaultI2PFile = getClass()
|
||||
.getClassLoader().getResource("defaults/i2p.properties");
|
||||
try {
|
||||
defaultI2PFile.withInputStream { i2pOptions.load(it) }
|
||||
} catch (ZipException mystery) {
|
||||
log.log(Level.SEVERE, "couldn't load default i2p properties", mystery)
|
||||
}
|
||||
defaultI2PFile.withInputStream { i2pOptions.load(it) }
|
||||
|
||||
def i2pOptionsFile = new File(home, "i2p.properties")
|
||||
if (i2pOptionsFile.exists()) {
|
||||
@ -173,17 +142,15 @@ public class Core {
|
||||
if (!i2pOptions.containsKey("outbound.nickname"))
|
||||
i2pOptions["outbound.nickname"] = "MuWire"
|
||||
}
|
||||
if (!(i2pOptions.containsKey("i2np.ntcp.port")
|
||||
&& i2pOptions.containsKey("i2np.udp.port")
|
||||
if (!(i2pOptions.hasProperty("i2np.ntcp.port")
|
||||
&& i2pOptions.hasProperty("i2np.udp.port")
|
||||
)) {
|
||||
Random r = new Random()
|
||||
int port = 9151 + r.nextInt(1 + 30777 - 9151) // this range matches what the i2p router would choose
|
||||
int port = r.nextInt(60000) + 4000
|
||||
i2pOptions["i2np.ntcp.port"] = String.valueOf(port)
|
||||
i2pOptions["i2np.udp.port"] = String.valueOf(port)
|
||||
i2pOptionsFile.withOutputStream { i2pOptions.store(it, "") }
|
||||
}
|
||||
|
||||
i2pOptions['i2cp.leaseSetEncType']='4,0'
|
||||
|
||||
if (!props.embeddedRouter) {
|
||||
if (!(I2PAppContext.getGlobalContext() instanceof RouterContext)) {
|
||||
@ -224,13 +191,14 @@ public class Core {
|
||||
|
||||
|
||||
// options like tunnel length and quantity
|
||||
I2PSocketManager socketManager
|
||||
keyDat.withInputStream {
|
||||
i2pSocketManager = new I2PSocketManagerFactory().createDisconnectedManager(it, i2pOptions["i2cp.tcp.host"], i2pOptions["i2cp.tcp.port"].toInteger(), i2pOptions)
|
||||
socketManager = new I2PSocketManagerFactory().createManager(it, i2pOptions["i2cp.tcp.host"], i2pOptions["i2cp.tcp.port"].toInteger(), i2pOptions)
|
||||
}
|
||||
i2pSocketManager.getDefaultOptions().setReadTimeout(60000)
|
||||
i2pSocketManager.getDefaultOptions().setConnectTimeout(30000)
|
||||
i2pSocketManager.addDisconnectListener({eventBus.publish(new RouterDisconnectedEvent())} as DisconnectListener)
|
||||
i2pSession = i2pSocketManager.getSession()
|
||||
socketManager.getDefaultOptions().setReadTimeout(60000)
|
||||
socketManager.getDefaultOptions().setConnectTimeout(30000)
|
||||
socketManager.addDisconnectListener({eventBus.publish(new RouterDisconnectedEvent())} as DisconnectListener)
|
||||
i2pSession = socketManager.getSession()
|
||||
|
||||
def destination = new Destination()
|
||||
spk = new SigningPrivateKey(Constants.SIG_TYPE)
|
||||
@ -287,22 +255,11 @@ public class Core {
|
||||
log.info("initializing mesh manager")
|
||||
MeshManager meshManager = new MeshManager(fileManager, home, props)
|
||||
eventBus.register(SourceDiscoveredEvent.class, meshManager)
|
||||
eventBus.register(SourceVerifiedEvent.class, meshManager)
|
||||
|
||||
log.info "initializing persistence service"
|
||||
persisterService = new PersisterService(new File(home, "files.json"), eventBus, 60000, fileManager)
|
||||
eventBus.register(UILoadedEvent.class, persisterService)
|
||||
|
||||
log.info "initializing folder persistence service"
|
||||
persisterFolderService = new PersisterFolderService(this, new File(home, "files"), eventBus)
|
||||
eventBus.register(PersisterDoneEvent.class, persisterFolderService)
|
||||
eventBus.register(FileDownloadedEvent.class, persisterFolderService)
|
||||
eventBus.register(FileLoadedEvent.class, persisterFolderService)
|
||||
eventBus.register(FileHashedEvent.class, persisterFolderService)
|
||||
eventBus.register(FileUnsharedEvent.class, persisterFolderService)
|
||||
eventBus.register(UICommentEvent.class, persisterFolderService)
|
||||
eventBus.register(UIFilePublishedEvent.class, persisterFolderService)
|
||||
eventBus.register(UIFileUnpublishedEvent.class, persisterFolderService)
|
||||
eventBus.register(UIPersistFilesEvent.class, persisterService)
|
||||
|
||||
log.info("initializing host cache")
|
||||
File hostStorage = new File(home, "hosts.json")
|
||||
@ -310,17 +267,10 @@ public class Core {
|
||||
eventBus.register(HostDiscoveredEvent.class, hostCache)
|
||||
eventBus.register(ConnectionEvent.class, hostCache)
|
||||
|
||||
|
||||
log.info("initializing responder cache")
|
||||
ResponderCache responderCache = new ResponderCache(props.responderCacheSize)
|
||||
eventBus.register(UIResultBatchEvent.class, responderCache)
|
||||
eventBus.register(SourceVerifiedEvent.class, responderCache)
|
||||
|
||||
|
||||
log.info("initializing connection manager")
|
||||
connectionManager = props.isLeaf() ?
|
||||
new LeafConnectionManager(eventBus, me, 3, hostCache, props) :
|
||||
new UltrapeerConnectionManager(eventBus, me, props.peerConnections, props.leafConnections, hostCache, responderCache, trustService, props)
|
||||
new UltrapeerConnectionManager(eventBus, me, 512, 512, hostCache, trustService, props)
|
||||
eventBus.register(TrustEvent.class, connectionManager)
|
||||
eventBus.register(ConnectionEvent.class, connectionManager)
|
||||
eventBus.register(DisconnectionEvent.class, connectionManager)
|
||||
@ -329,16 +279,16 @@ public class Core {
|
||||
log.info("initializing cache client")
|
||||
cacheClient = new CacheClient(eventBus,hostCache, connectionManager, i2pSession, props, 10000)
|
||||
|
||||
if (!(props.plugin || props.disableUpdates)) {
|
||||
if (!props.plugin) {
|
||||
log.info("initializing update client")
|
||||
updateClient = new UpdateClient(eventBus, i2pSession, myVersion, props, fileManager, me, spk)
|
||||
eventBus.register(FileDownloadedEvent.class, updateClient)
|
||||
eventBus.register(UIResultBatchEvent.class, updateClient)
|
||||
} else
|
||||
log.info("running as plugin or updates disabled, not initializing update client")
|
||||
log.info("running as plugin, not initializing update client")
|
||||
|
||||
log.info("initializing connector")
|
||||
I2PConnector i2pConnector = new I2PConnector(i2pSocketManager)
|
||||
I2PConnector i2pConnector = new I2PConnector(socketManager)
|
||||
|
||||
log.info("initializing certificate client")
|
||||
CertificateClient certificateClient = new CertificateClient(eventBus, i2pConnector)
|
||||
@ -352,19 +302,6 @@ public class Core {
|
||||
register(TrustEvent.class, chatServer)
|
||||
}
|
||||
|
||||
log.info("initializing feed manager")
|
||||
feedManager = new FeedManager(eventBus, home)
|
||||
eventBus.with {
|
||||
register(FeedItemFetchedEvent.class, feedManager)
|
||||
register(FeedFetchEvent.class, feedManager)
|
||||
register(UIFeedConfigurationEvent.class, feedManager)
|
||||
register(UIFeedDeletedEvent.class, feedManager)
|
||||
}
|
||||
|
||||
log.info("initializing feed client")
|
||||
feedClient = new FeedClient(i2pConnector, eventBus, me, feedManager)
|
||||
eventBus.register(UIFeedUpdateEvent.class, feedClient)
|
||||
|
||||
log.info "initializing results sender"
|
||||
ResultsSender resultsSender = new ResultsSender(eventBus, i2pConnector, me, props, certificateManager, chatServer)
|
||||
|
||||
@ -373,6 +310,23 @@ public class Core {
|
||||
eventBus.register(QueryEvent.class, searchManager)
|
||||
eventBus.register(ResultsEvent.class, searchManager)
|
||||
|
||||
log.info("initializing download manager")
|
||||
downloadManager = new DownloadManager(eventBus, trustService, meshManager, props, i2pConnector, home, me)
|
||||
eventBus.register(UIDownloadEvent.class, downloadManager)
|
||||
eventBus.register(UILoadedEvent.class, downloadManager)
|
||||
eventBus.register(FileDownloadedEvent.class, downloadManager)
|
||||
eventBus.register(UIDownloadCancelledEvent.class, downloadManager)
|
||||
eventBus.register(SourceDiscoveredEvent.class, downloadManager)
|
||||
eventBus.register(UIDownloadPausedEvent.class, downloadManager)
|
||||
eventBus.register(UIDownloadResumedEvent.class, downloadManager)
|
||||
|
||||
log.info("initializing upload manager")
|
||||
uploadManager = new UploadManager(eventBus, fileManager, meshManager, downloadManager, props)
|
||||
|
||||
log.info("initializing connection establisher")
|
||||
connectionEstablisher = new ConnectionEstablisher(eventBus, i2pConnector, props, connectionManager, hostCache)
|
||||
|
||||
|
||||
log.info("initializing chat manager")
|
||||
chatManager = new ChatManager(eventBus, me, i2pConnector, trustService, props)
|
||||
eventBus.with {
|
||||
@ -382,33 +336,17 @@ public class Core {
|
||||
register(ChatDisconnectionEvent.class, chatManager)
|
||||
}
|
||||
|
||||
log.info("initializing download manager")
|
||||
downloadManager = new DownloadManager(eventBus, trustService, meshManager, props, i2pConnector, home, me, chatServer)
|
||||
eventBus.register(UIDownloadEvent.class, downloadManager)
|
||||
eventBus.register(UIDownloadFeedItemEvent.class, downloadManager)
|
||||
eventBus.register(UILoadedEvent.class, downloadManager)
|
||||
eventBus.register(FileDownloadedEvent.class, downloadManager)
|
||||
eventBus.register(UIDownloadCancelledEvent.class, downloadManager)
|
||||
eventBus.register(SourceDiscoveredEvent.class, downloadManager)
|
||||
eventBus.register(UIDownloadPausedEvent.class, downloadManager)
|
||||
eventBus.register(UIDownloadResumedEvent.class, downloadManager)
|
||||
eventBus.register(DownloadHopelessEvent.class, downloadManager)
|
||||
|
||||
log.info("initializing upload manager")
|
||||
uploadManager = new UploadManager(eventBus, fileManager, meshManager, downloadManager, persisterFolderService, props)
|
||||
|
||||
log.info("initializing tracker responder")
|
||||
trackerResponder = new TrackerResponder(i2pSession, props, fileManager, downloadManager, meshManager, trustService, me)
|
||||
|
||||
log.info("initializing connection establisher")
|
||||
connectionEstablisher = new ConnectionEstablisher(eventBus, i2pConnector, props, connectionManager, hostCache)
|
||||
|
||||
log.info("initializing acceptor")
|
||||
I2PAcceptor i2pAcceptor = new I2PAcceptor(i2pSocketManager)
|
||||
I2PAcceptor i2pAcceptor = new I2PAcceptor(socketManager)
|
||||
connectionAcceptor = new ConnectionAcceptor(eventBus, connectionManager, props,
|
||||
i2pAcceptor, hostCache, trustService, searchManager, uploadManager, fileManager, connectionEstablisher,
|
||||
certificateManager, chatServer)
|
||||
|
||||
log.info("initializing directory watcher")
|
||||
directoryWatcher = new DirectoryWatcher(eventBus, fileManager, home, props)
|
||||
eventBus.register(DirectoryWatchedEvent.class, directoryWatcher)
|
||||
eventBus.register(AllFilesLoadedEvent.class, directoryWatcher)
|
||||
eventBus.register(DirectoryUnsharedEvent.class, directoryWatcher)
|
||||
|
||||
log.info("initializing hasher service")
|
||||
hasherService = new HasherService(new FileHasher(), eventBus, fileManager, props)
|
||||
@ -430,32 +368,9 @@ public class Core {
|
||||
BrowseManager browseManager = new BrowseManager(i2pConnector, eventBus, me)
|
||||
eventBus.register(UIBrowseEvent.class, browseManager)
|
||||
|
||||
log.info("initializing watched directory converter")
|
||||
watchedDirectoryConverter = new WatchedDirectoryConverter(this)
|
||||
eventBus.register(AllFilesLoadedEvent.class, watchedDirectoryConverter)
|
||||
|
||||
log.info("initializing watched directory manager")
|
||||
watchedDirectoryManager = new WatchedDirectoryManager(home, eventBus, fileManager)
|
||||
eventBus.with {
|
||||
register(WatchedDirectoryConfigurationEvent.class, watchedDirectoryManager)
|
||||
register(WatchedDirectoryConvertedEvent.class, watchedDirectoryManager)
|
||||
register(FileSharedEvent.class, watchedDirectoryManager)
|
||||
register(DirectoryUnsharedEvent.class, watchedDirectoryManager)
|
||||
register(UISyncDirectoryEvent.class, watchedDirectoryManager)
|
||||
}
|
||||
|
||||
log.info("initializing directory watcher")
|
||||
directoryWatcher = new DirectoryWatcher(eventBus, fileManager, home, watchedDirectoryManager)
|
||||
eventBus.with {
|
||||
register(DirectoryWatchedEvent.class, directoryWatcher)
|
||||
register(WatchedDirectoryConvertedEvent.class, directoryWatcher)
|
||||
register(DirectoryUnsharedEvent.class, directoryWatcher)
|
||||
register(WatchedDirectoryConfigurationEvent.class, directoryWatcher)
|
||||
}
|
||||
}
|
||||
|
||||
public void startServices() {
|
||||
i2pSession.connect()
|
||||
hasherService.start()
|
||||
trustService.start()
|
||||
trustService.waitForLoad()
|
||||
@ -466,9 +381,6 @@ public class Core {
|
||||
connectionEstablisher.start()
|
||||
hostCache.waitForLoad()
|
||||
updateClient?.start()
|
||||
feedManager.start()
|
||||
feedClient.start()
|
||||
trackerResponder.start()
|
||||
}
|
||||
|
||||
public void shutdown() {
|
||||
@ -486,8 +398,6 @@ public class Core {
|
||||
trustService.stop()
|
||||
log.info("shutting down persister service")
|
||||
persisterService.stop()
|
||||
log.info("shutting down persisterFolder service")
|
||||
persisterFolderService.stop()
|
||||
log.info("shutting down download manager")
|
||||
downloadManager.shutdown()
|
||||
log.info("shutting down connection acceptor")
|
||||
@ -496,28 +406,16 @@ public class Core {
|
||||
connectionEstablisher.stop()
|
||||
log.info("shutting down directory watcher")
|
||||
directoryWatcher.stop()
|
||||
log.info("shutting down watch directory manager")
|
||||
watchedDirectoryManager.shutdown()
|
||||
log.info("shutting down cache client")
|
||||
cacheClient.stop()
|
||||
log.info("shutting down chat server")
|
||||
chatServer.stop()
|
||||
log.info("shutting down chat manager")
|
||||
chatManager.shutdown()
|
||||
log.info("shutting down feed manager")
|
||||
feedManager.stop()
|
||||
log.info("shutting down feed client")
|
||||
feedClient.stop()
|
||||
log.info("shutting down tracker responder")
|
||||
trackerResponder.stop()
|
||||
log.info("shutting down connection manager")
|
||||
connectionManager.shutdown()
|
||||
if (updateClient != null) {
|
||||
log.info("shutting down update client")
|
||||
updateClient.stop()
|
||||
}
|
||||
log.info("killing socket manager")
|
||||
i2pSocketManager.destroySocketManager()
|
||||
log.info("killing i2p session")
|
||||
i2pSession.destroySession()
|
||||
if (router != null) {
|
||||
log.info("shutting down embedded router")
|
||||
router.shutdown(0)
|
||||
@ -561,7 +459,7 @@ public class Core {
|
||||
}
|
||||
}
|
||||
|
||||
Core core = new Core(props, home, "0.7.4")
|
||||
Core core = new Core(props, home, "0.6.8")
|
||||
core.startServices()
|
||||
|
||||
// ... at the end, sleep or execute script
|
||||
|
||||
@ -16,7 +16,7 @@ class MuWireSettings {
|
||||
boolean allowTrustLists
|
||||
int trustListInterval
|
||||
Set<Persona> trustSubscriptions
|
||||
int downloadRetryInterval, downloadMaxFailures
|
||||
int downloadRetryInterval
|
||||
int totalUploadSlots
|
||||
int uploadSlotsPerUser
|
||||
int updateCheckInterval
|
||||
@ -31,33 +31,17 @@ class MuWireSettings {
|
||||
boolean shareHiddenFiles
|
||||
boolean searchComments
|
||||
boolean browseFiles
|
||||
boolean allowTracking
|
||||
|
||||
boolean fileFeed
|
||||
boolean advertiseFeed
|
||||
boolean autoPublishSharedFiles
|
||||
boolean defaultFeedAutoDownload
|
||||
long defaultFeedUpdateInterval
|
||||
int defaultFeedItemsToKeep
|
||||
boolean defaultFeedSequential
|
||||
|
||||
int peerConnections
|
||||
int leafConnections
|
||||
|
||||
int responderCacheSize
|
||||
|
||||
boolean startChatServer
|
||||
int maxChatConnections
|
||||
boolean advertiseChat
|
||||
File chatWelcomeFile
|
||||
Set<String> watchedDirectories
|
||||
float downloadSequentialRatio
|
||||
int hostClearInterval, hostHopelessInterval, hostRejectInterval, hostHopelessPurgeInterval
|
||||
int hostClearInterval, hostHopelessInterval, hostRejectInterval
|
||||
int meshExpiration
|
||||
int speedSmoothSeconds
|
||||
boolean embeddedRouter
|
||||
boolean plugin
|
||||
boolean disableUpdates
|
||||
int inBw, outBw
|
||||
Set<String> watchedKeywords
|
||||
Set<String> watchedRegexes
|
||||
@ -81,7 +65,6 @@ class MuWireSettings {
|
||||
if (incompleteLocationProp != null)
|
||||
incompleteLocation = new File(incompleteLocationProp)
|
||||
downloadRetryInterval = Integer.parseInt(props.getProperty("downloadRetryInterval","60"))
|
||||
downloadMaxFailures = Integer.parseInt(props.getProperty("downloadMaxFailures","10"))
|
||||
updateCheckInterval = Integer.parseInt(props.getProperty("updateCheckInterval","24"))
|
||||
lastUpdateCheck = Long.parseLong(props.getProperty("lastUpdateChec","0"))
|
||||
autoDownloadUpdate = Boolean.parseBoolean(props.getProperty("autoDownloadUpdate","true"))
|
||||
@ -90,36 +73,16 @@ class MuWireSettings {
|
||||
shareHiddenFiles = Boolean.parseBoolean(props.getProperty("shareHiddenFiles","false"))
|
||||
downloadSequentialRatio = Float.valueOf(props.getProperty("downloadSequentialRatio","0.8"))
|
||||
hostClearInterval = Integer.valueOf(props.getProperty("hostClearInterval","15"))
|
||||
hostHopelessInterval = Integer.valueOf(props.getProperty("hostHopelessInterval", "60"))
|
||||
hostHopelessInterval = Integer.valueOf(props.getProperty("hostHopelessInterval", "1440"))
|
||||
hostRejectInterval = Integer.valueOf(props.getProperty("hostRejectInterval", "1"))
|
||||
hostHopelessPurgeInterval = Integer.valueOf(props.getProperty("hostHopelessPurgeInterval","1440"))
|
||||
meshExpiration = Integer.valueOf(props.getProperty("meshExpiration","60"))
|
||||
embeddedRouter = Boolean.valueOf(props.getProperty("embeddedRouter","false"))
|
||||
plugin = Boolean.valueOf(props.getProperty("plugin","false"))
|
||||
disableUpdates = Boolean.valueOf(props.getProperty("disableUpdates","false"))
|
||||
inBw = Integer.valueOf(props.getProperty("inBw","256"))
|
||||
outBw = Integer.valueOf(props.getProperty("outBw","128"))
|
||||
searchComments = Boolean.valueOf(props.getProperty("searchComments","true"))
|
||||
browseFiles = Boolean.valueOf(props.getProperty("browseFiles","true"))
|
||||
allowTracking = Boolean.valueOf(props.getProperty("allowTracking","true"))
|
||||
|
||||
// feed settings
|
||||
fileFeed = Boolean.valueOf(props.getProperty("fileFeed","true"))
|
||||
advertiseFeed = Boolean.valueOf(props.getProperty("advertiseFeed","true"))
|
||||
autoPublishSharedFiles = Boolean.valueOf(props.getProperty("autoPublishSharedFiles", "false"))
|
||||
defaultFeedAutoDownload = Boolean.valueOf(props.getProperty("defaultFeedAutoDownload", "false"))
|
||||
defaultFeedItemsToKeep = Integer.valueOf(props.getProperty("defaultFeedItemsToKeep", "1000"))
|
||||
defaultFeedSequential = Boolean.valueOf(props.getProperty("defaultFeedSequential", "false"))
|
||||
defaultFeedUpdateInterval = Long.valueOf(props.getProperty("defaultFeedUpdateInterval", "3600000"))
|
||||
|
||||
// ultrapeer connection settings
|
||||
leafConnections = Integer.valueOf(props.getProperty("leafConnections","512"))
|
||||
peerConnections = Integer.valueOf(props.getProperty("peerConnections","128"))
|
||||
|
||||
// responder cache settings
|
||||
responderCacheSize = Integer.valueOf(props.getProperty("responderCacheSize","32"))
|
||||
|
||||
speedSmoothSeconds = Integer.valueOf(props.getProperty("speedSmoothSeconds","10"))
|
||||
speedSmoothSeconds = Integer.valueOf(props.getProperty("speedSmoothSeconds","60"))
|
||||
totalUploadSlots = Integer.valueOf(props.getProperty("totalUploadSlots","-1"))
|
||||
uploadSlotsPerUser = Integer.valueOf(props.getProperty("uploadSlotsPerUser","-1"))
|
||||
startChatServer = Boolean.valueOf(props.getProperty("startChatServer","false"))
|
||||
@ -157,7 +120,6 @@ class MuWireSettings {
|
||||
if (incompleteLocation != null)
|
||||
props.setProperty("incompleteLocation", incompleteLocation.getAbsolutePath())
|
||||
props.setProperty("downloadRetryInterval", String.valueOf(downloadRetryInterval))
|
||||
props.setProperty("downloadMaxFailures", String.valueOf(downloadMaxFailures))
|
||||
props.setProperty("updateCheckInterval", String.valueOf(updateCheckInterval))
|
||||
props.setProperty("lastUpdateCheck", String.valueOf(lastUpdateCheck))
|
||||
props.setProperty("autoDownloadUpdate", String.valueOf(autoDownloadUpdate))
|
||||
@ -168,33 +130,13 @@ class MuWireSettings {
|
||||
props.setProperty("hostClearInterval", String.valueOf(hostClearInterval))
|
||||
props.setProperty("hostHopelessInterval", String.valueOf(hostHopelessInterval))
|
||||
props.setProperty("hostRejectInterval", String.valueOf(hostRejectInterval))
|
||||
props.setProperty("hostHopelessPurgeInterval", String.valueOf(hostHopelessPurgeInterval))
|
||||
props.setProperty("meshExpiration", String.valueOf(meshExpiration))
|
||||
props.setProperty("embeddedRouter", String.valueOf(embeddedRouter))
|
||||
props.setProperty("plugin", String.valueOf(plugin))
|
||||
props.setProperty("disableUpdates", String.valueOf(disableUpdates))
|
||||
props.setProperty("inBw", String.valueOf(inBw))
|
||||
props.setProperty("outBw", String.valueOf(outBw))
|
||||
props.setProperty("searchComments", String.valueOf(searchComments))
|
||||
props.setProperty("browseFiles", String.valueOf(browseFiles))
|
||||
props.setProperty("allowTracking", String.valueOf(allowTracking))
|
||||
|
||||
// feed settings
|
||||
props.setProperty("fileFeed", String.valueOf(fileFeed))
|
||||
props.setProperty("advertiseFeed", String.valueOf(advertiseFeed))
|
||||
props.setProperty("autoPublishSharedFiles", String.valueOf(autoPublishSharedFiles))
|
||||
props.setProperty("defaultFeedAutoDownload", String.valueOf(defaultFeedAutoDownload))
|
||||
props.setProperty("defaultFeedItemsToKeep", String.valueOf(defaultFeedItemsToKeep))
|
||||
props.setProperty("defaultFeedSequential", String.valueOf(defaultFeedSequential))
|
||||
props.setProperty("defaultFeedUpdateInterval", String.valueOf(defaultFeedUpdateInterval))
|
||||
|
||||
// ultrapeer connection settings
|
||||
props.setProperty("peerConnections", String.valueOf(peerConnections))
|
||||
props.setProperty("leafConnections", String.valueOf(leafConnections))
|
||||
|
||||
// responder cache settings
|
||||
props.setProperty("responderCacheSize", String.valueOf(responderCacheSize))
|
||||
|
||||
props.setProperty("speedSmoothSeconds", String.valueOf(speedSmoothSeconds))
|
||||
props.setProperty("totalUploadSlots", String.valueOf(totalUploadSlots))
|
||||
props.setProperty("uploadSlotsPerUser", String.valueOf(uploadSlotsPerUser))
|
||||
|
||||
@ -233,7 +233,7 @@ class ChatConnection implements ChatLink {
|
||||
daos.close()
|
||||
byte [] signed = baos.toByteArray()
|
||||
def spk = sender.destination.getSigningPublicKey()
|
||||
def signature = new Signature(spk.getType(), sig)
|
||||
def signature = new Signature(Constants.SIG_TYPE, sig)
|
||||
DSAEngine.getInstance().verifySignature(signature, signed, spk)
|
||||
}
|
||||
|
||||
|
||||
@ -29,10 +29,6 @@ class ChatManager {
|
||||
timer.schedule({connect()} as TimerTask, 1000, 1000)
|
||||
}
|
||||
|
||||
boolean isConnected(Persona p) {
|
||||
clients.containsKey(p)
|
||||
}
|
||||
|
||||
void onUIConnectChatEvent(UIConnectChatEvent e) {
|
||||
if (e.host == me) {
|
||||
eventBus.publish(new ChatConnectionEvent(status : ChatConnectionAttemptStatus.SUCCESSFUL,
|
||||
|
||||
@ -60,10 +60,6 @@ class ChatServer {
|
||||
echo(getWelcome(),me.destination)
|
||||
}
|
||||
|
||||
public boolean isRunning() {
|
||||
running.get()
|
||||
}
|
||||
|
||||
private String getWelcome() {
|
||||
String welcome = DEFAULT_WELCOME
|
||||
if (settings.chatWelcomeFile != null)
|
||||
@ -201,19 +197,9 @@ class ChatServer {
|
||||
return
|
||||
}
|
||||
|
||||
if (command.action.console && e.room != CONSOLE) {
|
||||
echo("/SAY ERROR: You can only execute that command in the chat console, not in a chat room.",
|
||||
e.sender.destination, e.room)
|
||||
return
|
||||
}
|
||||
|
||||
if (!command.action.console && e.room == CONSOLE) {
|
||||
echo("/SAY ERROR: You need to be in a chat room. Type /LIST for list of rooms or /JOIN to join or create a room.",
|
||||
e.sender.destination)
|
||||
return
|
||||
}
|
||||
|
||||
if (!command.action.user)
|
||||
if ((command.action.console && e.room != CONSOLE) ||
|
||||
(!command.action.console && e.room == CONSOLE) ||
|
||||
!command.action.user)
|
||||
return
|
||||
|
||||
if (command.action.local && e.sender != me)
|
||||
@ -310,17 +296,17 @@ class ChatServer {
|
||||
echo(help, d)
|
||||
}
|
||||
|
||||
private void echo(String payload, Destination d, String room = CONSOLE) {
|
||||
private void echo(String payload, Destination d) {
|
||||
log.info "echoing $payload"
|
||||
UUID uuid = UUID.randomUUID()
|
||||
long now = System.currentTimeMillis()
|
||||
byte [] sig = ChatConnection.sign(uuid, now, room, payload, me, me, spk)
|
||||
byte [] sig = ChatConnection.sign(uuid, now, CONSOLE, payload, me, me, spk)
|
||||
ChatMessageEvent echo = new ChatMessageEvent(
|
||||
uuid : uuid,
|
||||
payload : payload,
|
||||
sender : me,
|
||||
host : me,
|
||||
room : room,
|
||||
room : CONSOLE,
|
||||
chatTime : now,
|
||||
sig : sig
|
||||
)
|
||||
|
||||
@ -49,8 +49,6 @@ abstract class Connection implements Closeable {
|
||||
protected final String name
|
||||
|
||||
long lastPingSentTime, lastPongReceivedTime
|
||||
|
||||
private volatile UUID lastPingUUID
|
||||
|
||||
Connection(EventBus eventBus, Endpoint endpoint, boolean incoming,
|
||||
HostCache hostCache, TrustService trustService, MuWireSettings settings) {
|
||||
@ -134,8 +132,6 @@ abstract class Connection implements Closeable {
|
||||
def ping = [:]
|
||||
ping.type = "Ping"
|
||||
ping.version = 1
|
||||
lastPingUUID = UUID.randomUUID()
|
||||
ping.uuid = lastPingUUID.toString()
|
||||
messages.put(ping)
|
||||
lastPingSentTime = System.currentTimeMillis()
|
||||
}
|
||||
@ -164,14 +160,12 @@ abstract class Connection implements Closeable {
|
||||
messages.put(query)
|
||||
}
|
||||
|
||||
protected void handlePing(def ping) {
|
||||
protected void handlePing() {
|
||||
log.fine("$name received ping")
|
||||
def pong = [:]
|
||||
pong.type = "Pong"
|
||||
pong.version = 1
|
||||
if (ping.uuid != null)
|
||||
pong.uuid = ping.uuid
|
||||
pong.pongs = hostCache.getGoodHosts(2).collect { d -> d.toBase64() }
|
||||
pong.pongs = hostCache.getGoodHosts(10).collect { d -> d.toBase64() }
|
||||
messages.put(pong)
|
||||
}
|
||||
|
||||
@ -180,23 +174,7 @@ abstract class Connection implements Closeable {
|
||||
lastPongReceivedTime = System.currentTimeMillis()
|
||||
if (pong.pongs == null)
|
||||
throw new Exception("Pong doesn't have pongs")
|
||||
|
||||
if (lastPingUUID == null) {
|
||||
log.fine "$name received an unexpected pong"
|
||||
return
|
||||
}
|
||||
if (pong.uuid == null) {
|
||||
log.fine "$name pong doesn't have uuid"
|
||||
return
|
||||
}
|
||||
UUID pongUUID = UUID.fromString(pong.uuid)
|
||||
if (pongUUID != lastPingUUID) {
|
||||
log.fine "$name ping/pong uuid mismatch"
|
||||
return
|
||||
}
|
||||
lastPingUUID = null
|
||||
|
||||
pong.pongs.stream().limit(2).forEach {
|
||||
pong.pongs.each {
|
||||
def dest = new Destination(it)
|
||||
eventBus.publish(new HostDiscoveredEvent(destination: dest))
|
||||
}
|
||||
@ -266,7 +244,7 @@ abstract class Connection implements Closeable {
|
||||
else
|
||||
payload = String.join(" ",search.keywords).getBytes(StandardCharsets.UTF_8)
|
||||
def spk = originator.destination.getSigningPublicKey()
|
||||
def signature = new Signature(spk.getType(), sig)
|
||||
def signature = new Signature(Constants.SIG_TYPE, sig)
|
||||
if (!DSAEngine.getInstance().verifySignature(signature, payload, spk)) {
|
||||
log.info("signature didn't match keywords")
|
||||
return
|
||||
@ -277,6 +255,7 @@ abstract class Connection implements Closeable {
|
||||
return
|
||||
}
|
||||
|
||||
// TODO: make this mandatory at some point
|
||||
byte[] sig2 = null
|
||||
long queryTime = 0
|
||||
if (search.sig2 != null) {
|
||||
@ -288,7 +267,7 @@ abstract class Connection implements Closeable {
|
||||
queryTime = search.queryTime
|
||||
byte [] payload = (search.uuid + String.valueOf(queryTime)).getBytes(StandardCharsets.US_ASCII)
|
||||
def spk = originator.destination.getSigningPublicKey()
|
||||
def signature = new Signature(spk.getType(), sig2)
|
||||
def signature = new Signature(Constants.SIG_TYPE, sig2)
|
||||
if (!DSAEngine.getInstance().verifySignature(signature, payload, spk)) {
|
||||
log.info("extended signature didn't match uuid and timestamp")
|
||||
return
|
||||
@ -299,10 +278,8 @@ abstract class Connection implements Closeable {
|
||||
return
|
||||
}
|
||||
}
|
||||
} else {
|
||||
} else
|
||||
log.info("no extended signature in query")
|
||||
return
|
||||
}
|
||||
|
||||
SearchEvent searchEvent = new SearchEvent(searchTerms : search.keywords,
|
||||
searchHash : infohash,
|
||||
|
||||
@ -15,11 +15,9 @@ import com.muwire.core.EventBus
|
||||
import com.muwire.core.InfoHash
|
||||
import com.muwire.core.MuWireSettings
|
||||
import com.muwire.core.Persona
|
||||
import com.muwire.core.SharedFile
|
||||
import com.muwire.core.chat.ChatServer
|
||||
import com.muwire.core.filecert.Certificate
|
||||
import com.muwire.core.filecert.CertificateManager
|
||||
import com.muwire.core.filefeeds.FeedItems
|
||||
import com.muwire.core.files.FileManager
|
||||
import com.muwire.core.hostcache.HostCache
|
||||
import com.muwire.core.trust.TrustLevel
|
||||
@ -60,7 +58,7 @@ class ConnectionAcceptor {
|
||||
|
||||
private volatile shutdown
|
||||
|
||||
volatile int browsed
|
||||
private volatile int browsed
|
||||
|
||||
ConnectionAcceptor(EventBus eventBus, UltrapeerConnectionManager manager,
|
||||
MuWireSettings settings, I2PAcceptor acceptor, HostCache hostCache,
|
||||
@ -163,9 +161,6 @@ class ConnectionAcceptor {
|
||||
case (byte)'I':
|
||||
processIRC(e)
|
||||
break
|
||||
case (byte)'F':
|
||||
processFEED(e)
|
||||
break
|
||||
default:
|
||||
throw new Exception("Invalid read $read")
|
||||
}
|
||||
@ -315,9 +310,6 @@ class ConnectionAcceptor {
|
||||
boolean chat = false
|
||||
if (headers.containsKey('Chat'))
|
||||
chat = Boolean.parseBoolean(headers['Chat'])
|
||||
boolean feed = false
|
||||
if (headers.containsKey('Feed'))
|
||||
feed = Boolean.parseBoolean(headers['Feed'])
|
||||
|
||||
byte [] personaBytes = Base64.decode(headers['Sender'])
|
||||
Persona sender = new Persona(new ByteArrayInputStream(personaBytes))
|
||||
@ -337,7 +329,6 @@ class ConnectionAcceptor {
|
||||
def json = slurper.parse(payload)
|
||||
results[i] = ResultsParser.parse(sender, resultsUUID, json)
|
||||
results[i].chat = chat
|
||||
results[i].feed = feed
|
||||
}
|
||||
eventBus.publish(new UIResultBatchEvent(uuid: resultsUUID, results: results))
|
||||
} catch (IOException bad) {
|
||||
@ -380,19 +371,16 @@ class ConnectionAcceptor {
|
||||
|
||||
os.write("Count: ${sharedFiles.size()}\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
|
||||
boolean chat = chatServer.isRunning() && settings.advertiseChat
|
||||
boolean chat = chatServer.running.get() && settings.advertiseChat
|
||||
os.write("Chat: ${chat}\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
|
||||
boolean feed = settings.fileFeed && settings.advertiseFeed
|
||||
os.write("Feed: ${feed}\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
|
||||
os.write("\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
|
||||
DataOutputStream dos = new DataOutputStream(new GZIPOutputStream(os))
|
||||
JsonOutput jsonOutput = new JsonOutput()
|
||||
sharedFiles.each {
|
||||
it.hit(browser, System.currentTimeMillis(), "Browse Host");
|
||||
int certificates = certificateManager.getByInfoHash(new InfoHash(it.getRoot())).size()
|
||||
int certificates = certificateManager.getByInfoHash(it.getInfoHash()).size()
|
||||
def obj = ResultsSender.sharedFileToObj(it, false, certificates)
|
||||
def json = jsonOutput.toJson(obj)
|
||||
dos.writeShort((short)json.length())
|
||||
@ -536,58 +524,5 @@ class ConnectionAcceptor {
|
||||
throw new Exception("Invalid IRC connection")
|
||||
chatServer.handle(e)
|
||||
}
|
||||
|
||||
private void processFEED(Endpoint e) {
|
||||
try {
|
||||
byte[] EED = new byte[5];
|
||||
DataInputStream dis = new DataInputStream(e.getInputStream())
|
||||
dis.readFully(EED);
|
||||
if (EED != "EED\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
throw new Exception("Invalid FEED connection")
|
||||
|
||||
OutputStream os = e.getOutputStream()
|
||||
|
||||
Map<String, String> headers = DataUtil.readAllHeaders(dis)
|
||||
if (!headers.containsKey("Persona"))
|
||||
throw new Exception("Persona header missing")
|
||||
Persona requestor = new Persona(new ByteArrayInputStream(Base64.decode(headers['Persona'])))
|
||||
if (requestor.destination != e.destination)
|
||||
throw new Exception("Requestor persona mismatch")
|
||||
|
||||
if (!settings.fileFeed) {
|
||||
os.write("403 Not Allowed\r\n\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
os.flush()
|
||||
e.close()
|
||||
return
|
||||
}
|
||||
|
||||
long timestamp = 0
|
||||
if (headers.containsKey("Timestamp")) {
|
||||
timestamp = Long.parseLong(headers['Timestamp'])
|
||||
}
|
||||
|
||||
List<SharedFile> published = fileManager.getPublishedSince(timestamp)
|
||||
|
||||
os.write("200 OK\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
os.write("Count: ${published.size()}\r\n".getBytes(StandardCharsets.US_ASCII));
|
||||
os.write("\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
|
||||
DataOutputStream dos = new DataOutputStream(new GZIPOutputStream(os))
|
||||
JsonOutput jsonOutput = new JsonOutput()
|
||||
final long now = System.currentTimeMillis();
|
||||
published.each {
|
||||
it.hit(requestor, now, "Feed Update");
|
||||
int certificates = certificateManager.getByInfoHash(new InfoHash(it.getRoot())).size()
|
||||
def obj = FeedItems.sharedFileToObj(it, certificates)
|
||||
def json = jsonOutput.toJson(obj)
|
||||
dos.writeShort((short)json.length())
|
||||
dos.write(json.getBytes(StandardCharsets.US_ASCII))
|
||||
}
|
||||
dos.flush()
|
||||
dos.close()
|
||||
} finally {
|
||||
e.close()
|
||||
}
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@ -34,8 +34,6 @@ class ConnectionEstablisher {
|
||||
final ExecutorService executor, closer
|
||||
|
||||
final Set inProgress = new ConcurrentHashSet()
|
||||
|
||||
private volatile boolean shutdown
|
||||
|
||||
ConnectionEstablisher(){}
|
||||
|
||||
@ -62,15 +60,12 @@ class ConnectionEstablisher {
|
||||
}
|
||||
|
||||
void stop() {
|
||||
shutdown = true
|
||||
timer.cancel()
|
||||
executor.shutdownNow()
|
||||
closer.shutdownNow()
|
||||
}
|
||||
|
||||
private void connectIfNeeded() {
|
||||
if (shutdown)
|
||||
return
|
||||
if (!connectionManager.needsConnections())
|
||||
return
|
||||
if (inProgress.size() >= CONCURRENT)
|
||||
@ -94,8 +89,6 @@ class ConnectionEstablisher {
|
||||
}
|
||||
|
||||
private void connect(Destination toTry) {
|
||||
if (shutdown)
|
||||
return
|
||||
log.info("starting connect to ${toTry.toBase32()}")
|
||||
try {
|
||||
def endpoint = i2pConnector.connect(toTry)
|
||||
@ -130,8 +123,6 @@ class ConnectionEstablisher {
|
||||
}
|
||||
|
||||
private void fail(Endpoint endpoint) {
|
||||
if (shutdown)
|
||||
return
|
||||
if (!closer.isShutdown()) {
|
||||
closer.execute {
|
||||
endpoint.close()
|
||||
|
||||
@ -53,7 +53,7 @@ class PeerConnection extends Connection {
|
||||
if (json.type == null)
|
||||
throw new Exception("missing json type")
|
||||
switch(json.type) {
|
||||
case "Ping" : handlePing(json); break;
|
||||
case "Ping" : handlePing(); break;
|
||||
case "Pong" : handlePong(json); break;
|
||||
case "Search": handleSearch(json); break
|
||||
default :
|
||||
|
||||
@ -8,7 +8,6 @@ import com.muwire.core.MuWireSettings
|
||||
import com.muwire.core.Persona
|
||||
import com.muwire.core.hostcache.HostCache
|
||||
import com.muwire.core.search.QueryEvent
|
||||
import com.muwire.core.search.ResponderCache
|
||||
import com.muwire.core.trust.TrustService
|
||||
|
||||
import groovy.util.logging.Log
|
||||
@ -19,22 +18,18 @@ class UltrapeerConnectionManager extends ConnectionManager {
|
||||
|
||||
final int maxPeers, maxLeafs
|
||||
final TrustService trustService
|
||||
final ResponderCache responderCache
|
||||
|
||||
final Map<Destination, PeerConnection> peerConnections = new ConcurrentHashMap()
|
||||
final Map<Destination, LeafConnection> leafConnections = new ConcurrentHashMap()
|
||||
|
||||
private final Random random = new Random()
|
||||
|
||||
UltrapeerConnectionManager() {}
|
||||
|
||||
public UltrapeerConnectionManager(EventBus eventBus, Persona me, int maxPeers, int maxLeafs,
|
||||
HostCache hostCache, ResponderCache responderCache, TrustService trustService, MuWireSettings settings) {
|
||||
HostCache hostCache, TrustService trustService, MuWireSettings settings) {
|
||||
super(eventBus, me, hostCache, settings)
|
||||
this.maxPeers = maxPeers
|
||||
this.maxLeafs = maxLeafs
|
||||
this.trustService = trustService
|
||||
this.responderCache = responderCache
|
||||
}
|
||||
@Override
|
||||
public void drop(Destination d) {
|
||||
@ -49,18 +44,8 @@ class UltrapeerConnectionManager extends ConnectionManager {
|
||||
if (e.replyTo != me.destination && e.receivedOn != me.destination &&
|
||||
!leafConnections.containsKey(e.receivedOn))
|
||||
e.firstHop = false
|
||||
final int connCount = peerConnections.size()
|
||||
if (connCount == 0)
|
||||
return
|
||||
final int treshold = (int)(Math.sqrt(connCount)) + 1
|
||||
peerConnections.values().each {
|
||||
// 1. do not send query back to originator
|
||||
// 2. if firstHop forward to everyone
|
||||
// 3. otherwise to everyone who has recently responded/transferred to us + randomized sqrt of neighbors
|
||||
if (e.getReceivedOn() != it.getEndpoint().getDestination() &&
|
||||
(e.firstHop ||
|
||||
responderCache.hasResponded(it.endpoint.destination) ||
|
||||
random.nextInt(connCount) < treshold))
|
||||
if (e.getReceivedOn() != it.getEndpoint().getDestination())
|
||||
it.sendQuery(e)
|
||||
}
|
||||
}
|
||||
@ -120,8 +105,8 @@ class UltrapeerConnectionManager extends ConnectionManager {
|
||||
@Override
|
||||
void shutdown() {
|
||||
super.shutdown()
|
||||
peerConnections.values().stream().forEach({v -> v.close()})
|
||||
leafConnections.values().stream().forEach({v -> v.close()})
|
||||
peerConnections.values().stream().parallel().forEach({v -> v.close()})
|
||||
leafConnections.values().stream().parallel().forEach({v -> v.close()})
|
||||
peerConnections.clear()
|
||||
leafConnections.clear()
|
||||
}
|
||||
|
||||
@ -1,7 +0,0 @@
|
||||
package com.muwire.core.download
|
||||
|
||||
import com.muwire.core.Event
|
||||
|
||||
class DownloadHopelessEvent extends Event {
|
||||
Downloader downloader
|
||||
}
|
||||
@ -1,7 +1,6 @@
|
||||
package com.muwire.core.download
|
||||
|
||||
import com.muwire.core.connection.I2PConnector
|
||||
import com.muwire.core.filefeeds.UIDownloadFeedItemEvent
|
||||
import com.muwire.core.files.FileDownloadedEvent
|
||||
import com.muwire.core.files.FileHasher
|
||||
import com.muwire.core.mesh.Mesh
|
||||
@ -23,8 +22,6 @@ import com.muwire.core.InfoHash
|
||||
import com.muwire.core.MuWireSettings
|
||||
import com.muwire.core.Persona
|
||||
import com.muwire.core.UILoadedEvent
|
||||
import com.muwire.core.chat.ChatManager
|
||||
import com.muwire.core.chat.ChatServer
|
||||
|
||||
import java.util.concurrent.ConcurrentHashMap
|
||||
import java.util.concurrent.Executor
|
||||
@ -37,17 +34,16 @@ public class DownloadManager {
|
||||
private final EventBus eventBus
|
||||
private final TrustService trustService
|
||||
private final MeshManager meshManager
|
||||
final MuWireSettings muSettings
|
||||
private final MuWireSettings muSettings
|
||||
private final I2PConnector connector
|
||||
private final Executor executor
|
||||
private final File home
|
||||
private final Persona me
|
||||
private final ChatServer chatServer
|
||||
|
||||
private final Map<InfoHash, Downloader> downloaders = new ConcurrentHashMap<>()
|
||||
|
||||
public DownloadManager(EventBus eventBus, TrustService trustService, MeshManager meshManager, MuWireSettings muSettings,
|
||||
I2PConnector connector, File home, Persona me, ChatServer chatServer) {
|
||||
I2PConnector connector, File home, Persona me) {
|
||||
this.eventBus = eventBus
|
||||
this.trustService = trustService
|
||||
this.meshManager = meshManager
|
||||
@ -55,7 +51,6 @@ public class DownloadManager {
|
||||
this.connector = connector
|
||||
this.home = home
|
||||
this.me = me
|
||||
this.chatServer = chatServer
|
||||
|
||||
this.executor = Executors.newCachedThreadPool({ r ->
|
||||
Thread rv = new Thread(r)
|
||||
@ -67,6 +62,11 @@ public class DownloadManager {
|
||||
|
||||
|
||||
public void onUIDownloadEvent(UIDownloadEvent e) {
|
||||
|
||||
File incompletes = muSettings.incompleteLocation
|
||||
if (incompletes == null)
|
||||
incompletes = new File(home, "incompletes")
|
||||
incompletes.mkdirs()
|
||||
|
||||
def size = e.result[0].size
|
||||
def infohash = e.result[0].infohash
|
||||
@ -79,29 +79,12 @@ public class DownloadManager {
|
||||
destinations.addAll(e.sources)
|
||||
destinations.remove(me.destination)
|
||||
|
||||
doDownload(infohash, e.target, size, pieceSize, e.sequential, destinations)
|
||||
Pieces pieces = getPieces(infohash, size, pieceSize, e.sequential)
|
||||
|
||||
}
|
||||
|
||||
public void onUIDownloadFeedItemEvent(UIDownloadFeedItemEvent e) {
|
||||
Set<Destination> singleSource = new HashSet<>()
|
||||
singleSource.add(e.item.getPublisher().getDestination())
|
||||
doDownload(e.item.getInfoHash(), e.target, e.item.getSize(), e.item.getPieceSize(),
|
||||
e.sequential, singleSource)
|
||||
}
|
||||
|
||||
private void doDownload(InfoHash infoHash, File target, long size, int pieceSize,
|
||||
boolean sequential, Set<Destination> destinations) {
|
||||
File incompletes = muSettings.incompleteLocation
|
||||
if (incompletes == null)
|
||||
incompletes = new File(home, "incompletes")
|
||||
incompletes.mkdirs()
|
||||
|
||||
Pieces pieces = getPieces(infoHash, size, pieceSize, sequential)
|
||||
def downloader = new Downloader(eventBus, this, chatServer, me, target, size,
|
||||
infoHash, pieceSize, connector, destinations,
|
||||
incompletes, pieces, muSettings.downloadMaxFailures)
|
||||
downloaders.put(infoHash, downloader)
|
||||
def downloader = new Downloader(eventBus, this, me, e.target, size,
|
||||
infohash, pieceSize, connector, destinations,
|
||||
incompletes, pieces)
|
||||
downloaders.put(infohash, downloader)
|
||||
persistDownloaders()
|
||||
executor.execute({downloader.download()} as Runnable)
|
||||
eventBus.publish(new DownloadStartedEvent(downloader : downloader))
|
||||
@ -112,11 +95,6 @@ public class DownloadManager {
|
||||
persistDownloaders()
|
||||
}
|
||||
|
||||
public void onDownloadHopelessEvent(DownloadHopelessEvent e) {
|
||||
downloaders.remove(e.downloader.infoHash)
|
||||
persistDownloaders()
|
||||
}
|
||||
|
||||
public void onUIDownloadPausedEvent(UIDownloadPausedEvent e) {
|
||||
persistDownloaders()
|
||||
}
|
||||
@ -167,8 +145,8 @@ public class DownloadManager {
|
||||
|
||||
Pieces pieces = getPieces(infoHash, (long)json.length, json.pieceSizePow2, sequential)
|
||||
|
||||
def downloader = new Downloader(eventBus, this, chatServer, me, file, (long)json.length,
|
||||
infoHash, json.pieceSizePow2, connector, destinations, incompletes, pieces, muSettings.downloadMaxFailures)
|
||||
def downloader = new Downloader(eventBus, this, me, file, (long)json.length,
|
||||
infoHash, json.pieceSizePow2, connector, destinations, incompletes, pieces)
|
||||
if (json.paused != null)
|
||||
downloader.paused = json.paused
|
||||
|
||||
@ -251,8 +229,4 @@ public class DownloadManager {
|
||||
downloaders.values().each { it.stop() }
|
||||
Downloader.executorService.shutdownNow()
|
||||
}
|
||||
|
||||
public boolean isDownloading(InfoHash infoHash) {
|
||||
downloaders.containsKey(infoHash)
|
||||
}
|
||||
}
|
||||
|
||||
@ -37,15 +37,13 @@ class DownloadSession {
|
||||
private final long fileLength
|
||||
private final Set<Integer> available
|
||||
private final MessageDigest digest
|
||||
private final boolean browse, feed, chat
|
||||
|
||||
private final AtomicLong dataSinceLastRead
|
||||
|
||||
private MappedByteBuffer mapped
|
||||
|
||||
DownloadSession(EventBus eventBus, String meB64, Pieces pieces, InfoHash infoHash, Endpoint endpoint, File file,
|
||||
int pieceSize, long fileLength, Set<Integer> available, AtomicLong dataSinceLastRead,
|
||||
boolean browse, boolean feed, boolean chat) {
|
||||
int pieceSize, long fileLength, Set<Integer> available, AtomicLong dataSinceLastRead) {
|
||||
this.eventBus = eventBus
|
||||
this.meB64 = meB64
|
||||
this.pieces = pieces
|
||||
@ -56,9 +54,6 @@ class DownloadSession {
|
||||
this.fileLength = fileLength
|
||||
this.available = available
|
||||
this.dataSinceLastRead = dataSinceLastRead
|
||||
this.browse = browse
|
||||
this.feed = feed
|
||||
this.chat = chat
|
||||
try {
|
||||
digest = MessageDigest.getInstance("SHA-256")
|
||||
} catch (NoSuchAlgorithmException impossible) {
|
||||
@ -99,12 +94,6 @@ class DownloadSession {
|
||||
os.write("GET $root\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
os.write("Range: $start-$end\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
os.write("X-Persona: $meB64\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
if (browse)
|
||||
os.write("Browse: true\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
if (feed)
|
||||
os.write("Feed: true\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
if (chat)
|
||||
os.write("Chat: true\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
String xHave = DataUtil.encodeXHave(pieces.getDownloaded(), pieces.nPieces)
|
||||
os.write("X-Have: $xHave\r\n\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
os.flush()
|
||||
@ -194,14 +183,15 @@ class DownloadSession {
|
||||
mapped.position(position)
|
||||
|
||||
byte[] tmp = new byte[0x1 << 13]
|
||||
DataInputStream dis = new DataInputStream(is)
|
||||
while(mapped.hasRemaining()) {
|
||||
if (mapped.remaining() < tmp.length)
|
||||
tmp = new byte[mapped.remaining()]
|
||||
dis.readFully(tmp)
|
||||
int read = is.read(tmp)
|
||||
if (read == -1)
|
||||
throw new IOException()
|
||||
synchronized(this) {
|
||||
mapped.put(tmp)
|
||||
dataSinceLastRead.addAndGet(tmp.length)
|
||||
mapped.put(tmp, 0, read)
|
||||
dataSinceLastRead.addAndGet(read)
|
||||
pieces.markPartial(piece, mapped.position())
|
||||
}
|
||||
}
|
||||
@ -215,8 +205,6 @@ class DownloadSession {
|
||||
pieces.markPartial(piece, 0)
|
||||
throw new BadHashException("bad hash on piece $piece")
|
||||
}
|
||||
|
||||
eventBus.publish(new SourceVerifiedEvent(infoHash : infoHash, source : endpoint.destination))
|
||||
} finally {
|
||||
try { channel?.close() } catch (IOException ignore) {}
|
||||
DataUtil.tryUnmap(mapped)
|
||||
|
||||
@ -2,8 +2,6 @@ package com.muwire.core.download
|
||||
|
||||
import com.muwire.core.InfoHash
|
||||
import com.muwire.core.Persona
|
||||
import com.muwire.core.chat.ChatManager
|
||||
import com.muwire.core.chat.ChatServer
|
||||
import com.muwire.core.connection.Endpoint
|
||||
|
||||
import java.nio.file.AtomicMoveNotSupportedException
|
||||
@ -31,7 +29,7 @@ import net.i2p.util.ConcurrentHashSet
|
||||
@Log
|
||||
public class Downloader {
|
||||
|
||||
public enum DownloadState { CONNECTING, HASHLIST, DOWNLOADING, FAILED, HOPELESS, CANCELLED, PAUSED, FINISHED }
|
||||
public enum DownloadState { CONNECTING, HASHLIST, DOWNLOADING, FAILED, CANCELLED, PAUSED, FINISHED }
|
||||
private enum WorkerState { CONNECTING, HASHLIST, DOWNLOADING, FINISHED}
|
||||
|
||||
private static final ExecutorService executorService = Executors.newCachedThreadPool({r ->
|
||||
@ -43,7 +41,6 @@ public class Downloader {
|
||||
|
||||
private final EventBus eventBus
|
||||
private final DownloadManager downloadManager
|
||||
private final ChatServer chatServer
|
||||
private final Persona me
|
||||
private final File file
|
||||
private final Pieces pieces
|
||||
@ -59,14 +56,10 @@ public class Downloader {
|
||||
final int pieceSizePow2
|
||||
private final Map<Destination, DownloadWorker> activeWorkers = new ConcurrentHashMap<>()
|
||||
private final Set<Destination> successfulDestinations = new ConcurrentHashSet<>()
|
||||
/** LOCKING: itself */
|
||||
private final Map<Destination, Integer> failingDestinations = new HashMap<>()
|
||||
private final int maxFailures
|
||||
|
||||
|
||||
private volatile boolean cancelled, paused
|
||||
private final AtomicBoolean eventFired = new AtomicBoolean()
|
||||
private final AtomicBoolean hopelessEventFired = new AtomicBoolean()
|
||||
private boolean piecesFileClosed
|
||||
|
||||
private final AtomicLong dataSinceLastRead = new AtomicLong(0)
|
||||
@ -75,14 +68,13 @@ public class Downloader {
|
||||
private int speedPos = 0
|
||||
private int speedAvg = 0
|
||||
|
||||
public Downloader(EventBus eventBus, DownloadManager downloadManager, ChatServer chatServer,
|
||||
public Downloader(EventBus eventBus, DownloadManager downloadManager,
|
||||
Persona me, File file, long length, InfoHash infoHash,
|
||||
int pieceSizePow2, I2PConnector connector, Set<Destination> destinations,
|
||||
File incompletes, Pieces pieces, int maxFailures) {
|
||||
File incompletes, Pieces pieces) {
|
||||
this.eventBus = eventBus
|
||||
this.me = me
|
||||
this.downloadManager = downloadManager
|
||||
this.chatServer = chatServer
|
||||
this.file = file
|
||||
this.infoHash = infoHash
|
||||
this.length = length
|
||||
@ -95,7 +87,6 @@ public class Downloader {
|
||||
this.pieceSize = 1 << pieceSizePow2
|
||||
this.pieces = pieces
|
||||
this.nPieces = pieces.nPieces
|
||||
this.maxFailures = maxFailures
|
||||
}
|
||||
|
||||
public synchronized InfoHash getInfoHash() {
|
||||
@ -125,7 +116,7 @@ public class Downloader {
|
||||
void download() {
|
||||
readPieces()
|
||||
destinations.each {
|
||||
if (it != me.destination && !isHopeless(it)) {
|
||||
if (it != me.destination) {
|
||||
def worker = new DownloadWorker(it)
|
||||
activeWorkers.put(it, worker)
|
||||
executorService.submit(worker)
|
||||
@ -169,7 +160,7 @@ public class Downloader {
|
||||
long dataRead = dataSinceLastRead.getAndSet(0)
|
||||
long now = System.currentTimeMillis()
|
||||
if (now > lastSpeedRead)
|
||||
currSpeed = (int) (dataRead * 1000.0d / (now - lastSpeedRead))
|
||||
currSpeed = (int) (dataRead * 1000.0 / (now - lastSpeedRead))
|
||||
lastSpeedRead = now
|
||||
}
|
||||
|
||||
@ -215,8 +206,6 @@ public class Downloader {
|
||||
if (allFinished) {
|
||||
if (pieces.isComplete())
|
||||
return DownloadState.FINISHED
|
||||
if (!hasLiveSources())
|
||||
return DownloadState.HOPELESS
|
||||
return DownloadState.FAILED
|
||||
}
|
||||
|
||||
@ -280,22 +269,11 @@ public class Downloader {
|
||||
public int getTotalWorkers() {
|
||||
return activeWorkers.size();
|
||||
}
|
||||
|
||||
public int countHopelessSources() {
|
||||
synchronized(failingDestinations) {
|
||||
return destinations.count { isHopeless(it)}
|
||||
}
|
||||
}
|
||||
|
||||
private boolean hasLiveSources() {
|
||||
destinations.size() > countHopelessSources()
|
||||
}
|
||||
|
||||
public void resume() {
|
||||
paused = false
|
||||
readPieces()
|
||||
destinations.stream().filter({!isHopeless(it)}).forEach { destination ->
|
||||
log.fine("resuming source ${destination.toBase32()}")
|
||||
destinations.each { destination ->
|
||||
def worker = activeWorkers.get(destination)
|
||||
if (worker != null) {
|
||||
if (worker.currentState == WorkerState.FINISHED) {
|
||||
@ -312,9 +290,8 @@ public class Downloader {
|
||||
}
|
||||
|
||||
void addSource(Destination d) {
|
||||
if (activeWorkers.containsKey(d) || isHopeless(d))
|
||||
if (activeWorkers.containsKey(d))
|
||||
return
|
||||
destinations.add(d)
|
||||
DownloadWorker newWorker = new DownloadWorker(d)
|
||||
activeWorkers.put(d, newWorker)
|
||||
executorService.submit(newWorker)
|
||||
@ -370,28 +347,6 @@ public class Downloader {
|
||||
try {os?.close() } catch (IOException ignore) {}
|
||||
}
|
||||
}
|
||||
|
||||
private boolean isHopeless(Destination d) {
|
||||
if (maxFailures < 0)
|
||||
return false
|
||||
synchronized(failingDestinations) {
|
||||
return !successfulDestinations.contains(d) &&
|
||||
failingDestinations.containsKey(d) &&
|
||||
failingDestinations[d] >= maxFailures
|
||||
}
|
||||
}
|
||||
|
||||
private void markFailed(Destination d) {
|
||||
log.fine("marking failed ${d.toBase32()}")
|
||||
synchronized(failingDestinations) {
|
||||
Integer count = failingDestinations.get(d)
|
||||
if (count == null) {
|
||||
failingDestinations.put(d, 1)
|
||||
} else {
|
||||
failingDestinations.put(d, count + 1)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
class DownloadWorker implements Runnable {
|
||||
private final Destination destination
|
||||
@ -418,16 +373,10 @@ public class Downloader {
|
||||
setInfoHash(received)
|
||||
}
|
||||
currentState = WorkerState.DOWNLOADING
|
||||
|
||||
boolean browse = downloadManager.muSettings.browseFiles
|
||||
boolean feed = downloadManager.muSettings.fileFeed && downloadManager.muSettings.advertiseFeed
|
||||
boolean chat = chatServer.isRunning() && downloadManager.muSettings.advertiseChat
|
||||
|
||||
boolean requestPerformed
|
||||
while(!pieces.isComplete()) {
|
||||
currentSession = new DownloadSession(eventBus, me.toBase64(), pieces, getInfoHash(),
|
||||
endpoint, incompleteFile, pieceSize, length, available, dataSinceLastRead,
|
||||
browse, feed, chat)
|
||||
endpoint, incompleteFile, pieceSize, length, available, dataSinceLastRead)
|
||||
requestPerformed = currentSession.request()
|
||||
if (!requestPerformed)
|
||||
break
|
||||
@ -436,9 +385,6 @@ public class Downloader {
|
||||
}
|
||||
} catch (Exception bad) {
|
||||
log.log(Level.WARNING,"Exception while downloading",DataUtil.findRoot(bad))
|
||||
markFailed(destination)
|
||||
if (!hasLiveSources() && hopelessEventFired.compareAndSet(false, true))
|
||||
eventBus.publish(new DownloadHopelessEvent(downloader : Downloader.this))
|
||||
} finally {
|
||||
writePieces()
|
||||
currentState = WorkerState.FINISHED
|
||||
@ -459,9 +405,8 @@ public class Downloader {
|
||||
}
|
||||
eventBus.publish(
|
||||
new FileDownloadedEvent(
|
||||
downloadedFile : new DownloadedFile(file.getCanonicalFile(), getInfoHash().getRoot(), pieceSizePow2, successfulDestinations),
|
||||
downloader : Downloader.this,
|
||||
infoHash: getInfoHash()))
|
||||
downloadedFile : new DownloadedFile(file.getCanonicalFile(), getInfoHash(), pieceSizePow2, successfulDestinations),
|
||||
downloader : Downloader.this))
|
||||
|
||||
}
|
||||
endpoint?.close()
|
||||
|
||||
@ -2,11 +2,10 @@ package com.muwire.core.download
|
||||
|
||||
class Pieces {
|
||||
private final BitSet done, claimed
|
||||
final int nPieces
|
||||
private final int nPieces
|
||||
private final float ratio
|
||||
private final Random random = new Random()
|
||||
private final Map<Integer,Integer> partials = new HashMap<>()
|
||||
private int cachedDone;
|
||||
|
||||
Pieces(int nPieces) {
|
||||
this(nPieces, 1.0f)
|
||||
@ -79,7 +78,6 @@ class Pieces {
|
||||
if (piece >= nPieces)
|
||||
throw new IllegalArgumentException("invalid piece marked as downloaded? $piece/$nPieces")
|
||||
done.set(piece)
|
||||
cachedDone = done.cardinality();
|
||||
claimed.set(piece)
|
||||
partials.remove(piece)
|
||||
}
|
||||
@ -93,11 +91,11 @@ class Pieces {
|
||||
}
|
||||
|
||||
synchronized boolean isComplete() {
|
||||
cachedDone == nPieces
|
||||
done.cardinality() == nPieces
|
||||
}
|
||||
|
||||
synchronized int donePieces() {
|
||||
cachedDone
|
||||
done.cardinality()
|
||||
}
|
||||
|
||||
synchronized boolean isDownloaded(int piece) {
|
||||
@ -106,7 +104,6 @@ class Pieces {
|
||||
|
||||
synchronized void clearAll() {
|
||||
done.clear()
|
||||
cachedDone = 0
|
||||
claimed.clear()
|
||||
partials.clear()
|
||||
}
|
||||
|
||||
@ -1,11 +0,0 @@
|
||||
package com.muwire.core.download
|
||||
|
||||
import com.muwire.core.Event
|
||||
import com.muwire.core.InfoHash
|
||||
|
||||
import net.i2p.data.Destination
|
||||
|
||||
class SourceVerifiedEvent extends Event {
|
||||
InfoHash infoHash
|
||||
Destination source
|
||||
}
|
||||
@ -105,7 +105,7 @@ class Certificate {
|
||||
|
||||
byte [] payload = baos.toByteArray()
|
||||
SigningPublicKey spk = issuer.destination.getSigningPublicKey()
|
||||
Signature signature = new Signature(spk.getType(), sig)
|
||||
Signature signature = new Signature(Constants.SIG_TYPE, sig)
|
||||
DSAEngine.getInstance().verifySignature(signature, payload, spk)
|
||||
}
|
||||
|
||||
|
||||
@ -70,7 +70,7 @@ class CertificateManager {
|
||||
}
|
||||
|
||||
void onUICreateCertificateEvent(UICreateCertificateEvent e) {
|
||||
InfoHash infoHash = new InfoHash(e.sharedFile.getRoot())
|
||||
InfoHash infoHash = e.sharedFile.getInfoHash()
|
||||
String name = e.sharedFile.getFile().getName()
|
||||
long timestamp = System.currentTimeMillis()
|
||||
|
||||
|
||||
@ -1,110 +0,0 @@
|
||||
package com.muwire.core.filefeeds
|
||||
|
||||
import java.util.logging.Level
|
||||
import java.nio.charset.StandardCharsets
|
||||
import java.util.concurrent.Executor
|
||||
import java.util.concurrent.ExecutorService
|
||||
import java.util.concurrent.Executors
|
||||
import java.util.zip.GZIPInputStream
|
||||
|
||||
import com.muwire.core.EventBus
|
||||
import com.muwire.core.Persona
|
||||
import com.muwire.core.connection.Endpoint
|
||||
import com.muwire.core.connection.I2PConnector
|
||||
import com.muwire.core.util.DataUtil
|
||||
|
||||
import groovy.json.JsonSlurper
|
||||
import groovy.util.logging.Log
|
||||
|
||||
@Log
|
||||
class FeedClient {
|
||||
|
||||
private final I2PConnector connector
|
||||
private final EventBus eventBus
|
||||
private final Persona me
|
||||
private final FeedManager feedManager
|
||||
|
||||
private final ExecutorService feedFetcher = Executors.newCachedThreadPool()
|
||||
private final Timer feedUpdater = new Timer("feed-updater", true)
|
||||
|
||||
FeedClient(I2PConnector connector, EventBus eventBus, Persona me, FeedManager feedManager) {
|
||||
this.connector = connector
|
||||
this.eventBus = eventBus
|
||||
this.me = me
|
||||
this.feedManager = feedManager
|
||||
}
|
||||
|
||||
private void start() {
|
||||
feedUpdater.schedule({updateAnyFeeds()} as TimerTask, 60000, 60000)
|
||||
}
|
||||
|
||||
private void stop() {
|
||||
feedUpdater.cancel()
|
||||
feedFetcher.shutdown()
|
||||
}
|
||||
|
||||
private void updateAnyFeeds() {
|
||||
feedManager.getFeedsToUpdate().each { feed ->
|
||||
feedFetcher.execute({updateFeed(feed)} as Runnable)
|
||||
}
|
||||
}
|
||||
|
||||
void onUIFeedUpdateEvent(UIFeedUpdateEvent e) {
|
||||
Feed feed = feedManager.getFeed(e.host)
|
||||
if (feed == null) {
|
||||
log.severe("UI request to update non-existent feed " + e.host.getHumanReadableName())
|
||||
return
|
||||
}
|
||||
|
||||
feedFetcher.execute({updateFeed(feed)} as Runnable)
|
||||
}
|
||||
|
||||
private void updateFeed(Feed feed) {
|
||||
log.info("updating feed " + feed.getPublisher().getHumanReadableName())
|
||||
Endpoint endpoint = null
|
||||
try {
|
||||
eventBus.publish(new FeedFetchEvent(host : feed.getPublisher(), status : FeedFetchStatus.CONNECTING))
|
||||
feed.setLastUpdateAttempt(System.currentTimeMillis())
|
||||
endpoint = connector.connect(feed.getPublisher().getDestination())
|
||||
OutputStream os = endpoint.getOutputStream()
|
||||
os.write("FEED\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
os.write("Persona:${me.toBase64()}\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
os.write("Timestamp:${feed.getLastUpdated()}\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
os.write("\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
os.flush()
|
||||
|
||||
InputStream is = endpoint.getInputStream()
|
||||
String code = DataUtil.readTillRN(is)
|
||||
if (!code.startsWith("200"))
|
||||
throw new IOException("Invalid code $code")
|
||||
|
||||
// parse all headers
|
||||
Map<String,String> headers = DataUtil.readAllHeaders(is)
|
||||
|
||||
if (!headers.containsKey("Count"))
|
||||
throw new IOException("No count header")
|
||||
|
||||
int items = Integer.parseInt(headers['Count'])
|
||||
|
||||
eventBus.publish(new FeedFetchEvent(host : feed.getPublisher(), status : FeedFetchStatus.FETCHING, totalItems: items))
|
||||
|
||||
JsonSlurper slurper = new JsonSlurper()
|
||||
DataInputStream dis = new DataInputStream(new GZIPInputStream(is))
|
||||
for (int i = 0; i < items; i++) {
|
||||
int size = dis.readUnsignedShort()
|
||||
byte [] tmp = new byte[size]
|
||||
dis.readFully(tmp)
|
||||
def json = slurper.parse(tmp)
|
||||
FeedItem item = FeedItems.objToFeedItem(json, feed.getPublisher())
|
||||
eventBus.publish(new FeedItemFetchedEvent(item: item))
|
||||
}
|
||||
|
||||
eventBus.publish(new FeedFetchEvent(host : feed.getPublisher(), status : FeedFetchStatus.FINISHED))
|
||||
} catch (Exception bad) {
|
||||
log.log(Level.WARNING, "Feed update failed", bad)
|
||||
eventBus.publish(new FeedFetchEvent(host : feed.getPublisher(), status : FeedFetchStatus.FAILED))
|
||||
} finally {
|
||||
endpoint?.close()
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -1,10 +0,0 @@
|
||||
package com.muwire.core.filefeeds
|
||||
|
||||
import com.muwire.core.Event
|
||||
import com.muwire.core.Persona
|
||||
|
||||
class FeedFetchEvent extends Event {
|
||||
Persona host
|
||||
FeedFetchStatus status
|
||||
int totalItems
|
||||
}
|
||||
@ -1,7 +0,0 @@
|
||||
package com.muwire.core.filefeeds
|
||||
|
||||
import com.muwire.core.Event
|
||||
|
||||
class FeedItemFetchedEvent extends Event {
|
||||
FeedItem item
|
||||
}
|
||||
@ -1,7 +0,0 @@
|
||||
package com.muwire.core.filefeeds
|
||||
|
||||
import com.muwire.core.Event
|
||||
|
||||
class FeedItemLoadedEvent extends Event {
|
||||
FeedItem item
|
||||
}
|
||||
@ -1,79 +0,0 @@
|
||||
package com.muwire.core.filefeeds
|
||||
|
||||
import com.muwire.core.InfoHash
|
||||
import com.muwire.core.Persona
|
||||
import com.muwire.core.SharedFile
|
||||
import com.muwire.core.files.FileHasher
|
||||
import com.muwire.core.util.DataUtil
|
||||
|
||||
import net.i2p.data.Base64
|
||||
|
||||
class FeedItems {
|
||||
|
||||
public static def sharedFileToObj(SharedFile sf, int certificates) {
|
||||
def json = [:]
|
||||
json.type = "FeedItem"
|
||||
json.version = 1
|
||||
json.name = Base64.encode(DataUtil.encodei18nString(sf.getFile().getName()))
|
||||
json.infoHash = Base64.encode(sf.getRoot())
|
||||
json.size = sf.getCachedLength()
|
||||
json.pieceSize = sf.getPieceSize()
|
||||
|
||||
if (sf.getComment() != null)
|
||||
json.comment = sf.getComment()
|
||||
|
||||
json.certificates = certificates
|
||||
|
||||
json.timestamp = sf.getPublishedTimestamp()
|
||||
|
||||
json
|
||||
}
|
||||
|
||||
public static FeedItem objToFeedItem(def obj, Persona publisher) throws InvalidFeedItemException {
|
||||
if (obj.timestamp == null)
|
||||
throw new InvalidFeedItemException("No timestamp");
|
||||
if (obj.name == null)
|
||||
throw new InvalidFeedItemException("No name");
|
||||
if (obj.size == null || obj.size <= 0 || obj.size > FileHasher.MAX_SIZE)
|
||||
throw new InvalidFeedItemException("length missing or invalid ${obj.size}")
|
||||
if (obj.pieceSize == null || obj.pieceSize < FileHasher.MIN_PIECE_SIZE_POW2 || obj.pieceSize > FileHasher.MAX_PIECE_SIZE_POW2)
|
||||
throw new InvalidFeedItemException("piece size missing or invalid ${obj.pieceSize}")
|
||||
if (obj.infoHash == null)
|
||||
throw new InvalidFeedItemException("Infohash missing")
|
||||
|
||||
|
||||
InfoHash infoHash
|
||||
try {
|
||||
infoHash = new InfoHash(Base64.decode(obj.infoHash))
|
||||
} catch (Exception bad) {
|
||||
throw new InvalidFeedItemException("Invalid infohash", bad)
|
||||
}
|
||||
|
||||
String name
|
||||
try {
|
||||
name = DataUtil.readi18nString(Base64.decode(obj.name))
|
||||
} catch (Exception bad) {
|
||||
throw new InvalidFeedItemException("Invalid name", bad)
|
||||
}
|
||||
|
||||
int certificates = 0
|
||||
if (obj.certificates != null)
|
||||
certificates = obj.certificates
|
||||
|
||||
new FeedItem(publisher, obj.timestamp, name, obj.size, obj.pieceSize, infoHash, certificates, obj.comment)
|
||||
}
|
||||
|
||||
public static def feedItemToObj(FeedItem item) {
|
||||
def json = [:]
|
||||
json.type = "FeedItem"
|
||||
json.version = 1
|
||||
json.name = Base64.encode(DataUtil.encodei18nString(item.getName()))
|
||||
json.infoHash = Base64.encode(item.getInfoHash().getRoot())
|
||||
json.size = item.getSize()
|
||||
json.pieceSize = item.getPieceSize()
|
||||
json.timestamp = item.getTimestamp()
|
||||
json.certificates = item.getCertificates()
|
||||
json.comment = item.getComment()
|
||||
json
|
||||
}
|
||||
}
|
||||
@ -1,7 +0,0 @@
|
||||
package com.muwire.core.filefeeds
|
||||
|
||||
import com.muwire.core.Event
|
||||
|
||||
class FeedLoadedEvent extends Event {
|
||||
Feed feed
|
||||
}
|
||||
@ -1,225 +0,0 @@
|
||||
package com.muwire.core.filefeeds
|
||||
|
||||
import java.nio.file.Files
|
||||
import java.util.concurrent.ConcurrentHashMap
|
||||
import java.util.concurrent.ExecutorService
|
||||
import java.util.concurrent.Executors
|
||||
import java.util.concurrent.ThreadFactory
|
||||
import java.util.stream.Collectors
|
||||
|
||||
import com.muwire.core.EventBus
|
||||
import com.muwire.core.Persona
|
||||
|
||||
import groovy.json.JsonOutput
|
||||
import groovy.json.JsonSlurper
|
||||
import groovy.util.logging.Log
|
||||
|
||||
import net.i2p.data.Base64
|
||||
import net.i2p.util.ConcurrentHashSet
|
||||
|
||||
@Log
|
||||
class FeedManager {
|
||||
|
||||
private final EventBus eventBus
|
||||
private final File metadataFolder, itemsFolder
|
||||
private final Map<Persona, Feed> feeds = new ConcurrentHashMap<>()
|
||||
private final Map<Persona, Set<FeedItem>> feedItems = new ConcurrentHashMap<>()
|
||||
|
||||
private final ExecutorService persister = Executors.newSingleThreadExecutor({r ->
|
||||
new Thread(r, "feed persister")
|
||||
} as ThreadFactory)
|
||||
|
||||
|
||||
FeedManager(EventBus eventBus, File home) {
|
||||
this.eventBus = eventBus
|
||||
File feedsFolder = new File(home, "filefeeds")
|
||||
if (!feedsFolder.exists())
|
||||
feedsFolder.mkdir()
|
||||
this.metadataFolder = new File(feedsFolder, "metadata")
|
||||
if (!metadataFolder.exists())
|
||||
metadataFolder.mkdir()
|
||||
this.itemsFolder = new File(feedsFolder, "items")
|
||||
if (!itemsFolder.exists())
|
||||
itemsFolder.mkdir()
|
||||
}
|
||||
|
||||
public Feed getFeed(Persona persona) {
|
||||
feeds.get(persona)
|
||||
}
|
||||
|
||||
public Set<FeedItem> getFeedItems(Persona persona) {
|
||||
feedItems.getOrDefault(persona, Collections.emptySet())
|
||||
}
|
||||
|
||||
public List<Feed> getFeedsToUpdate() {
|
||||
long now = System.currentTimeMillis()
|
||||
feeds.values().stream().
|
||||
filter({Feed f -> !f.getStatus().isActive()}).
|
||||
filter({Feed f -> f.getLastUpdateAttempt() + f.getUpdateInterval() <= now})
|
||||
.collect(Collectors.toList())
|
||||
}
|
||||
|
||||
void start() {
|
||||
log.info("starting feed manager")
|
||||
persister.submit({loadFeeds()} as Runnable)
|
||||
persister.submit({loadItems()} as Runnable)
|
||||
}
|
||||
|
||||
void stop() {
|
||||
persister.shutdown()
|
||||
}
|
||||
|
||||
private void loadFeeds() {
|
||||
def slurper = new JsonSlurper()
|
||||
Files.walk(metadataFolder.toPath()).
|
||||
filter( { it.getFileName().toString().endsWith(".json")}).
|
||||
forEach( {
|
||||
def parsed = slurper.parse(it.toFile())
|
||||
Persona publisher = new Persona(new ByteArrayInputStream(Base64.decode(parsed.publisher)))
|
||||
Feed feed = new Feed(publisher)
|
||||
feed.setUpdateInterval(parsed.updateInterval)
|
||||
feed.setLastUpdated(parsed.lastUpdated)
|
||||
feed.setLastUpdateAttempt(parsed.lastUpdateAttempt)
|
||||
feed.setItemsToKeep(parsed.itemsToKeep)
|
||||
feed.setAutoDownload(parsed.autoDownload)
|
||||
feed.setSequential(parsed.sequential)
|
||||
|
||||
feed.setStatus(FeedFetchStatus.IDLE)
|
||||
|
||||
feeds.put(feed.getPublisher(), feed)
|
||||
|
||||
eventBus.publish(new FeedLoadedEvent(feed : feed))
|
||||
})
|
||||
}
|
||||
|
||||
private void loadItems() {
|
||||
def slurper = new JsonSlurper()
|
||||
feeds.keySet().each { persona ->
|
||||
File itemsFile = getItemsFile(feeds[persona])
|
||||
if (!itemsFile.exists())
|
||||
return // no items yet?
|
||||
itemsFile.eachLine { line ->
|
||||
def parsed = slurper.parseText(line)
|
||||
FeedItem item = FeedItems.objToFeedItem(parsed, persona)
|
||||
Set<FeedItem> items = feedItems.get(persona)
|
||||
if (items == null) {
|
||||
items = new ConcurrentHashSet<>()
|
||||
feedItems.put(persona, items)
|
||||
}
|
||||
items.add(item)
|
||||
eventBus.publish(new FeedItemLoadedEvent(item : item))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
void onFeedItemFetchedEvent(FeedItemFetchedEvent e) {
|
||||
Set<FeedItem> set = feedItems.get(e.item.getPublisher())
|
||||
if (set == null) {
|
||||
set = new ConcurrentHashSet<>()
|
||||
feedItems.put(e.getItem().getPublisher(), set)
|
||||
}
|
||||
set.add(e.item)
|
||||
}
|
||||
|
||||
void onFeedFetchEvent(FeedFetchEvent e) {
|
||||
|
||||
Feed feed = feeds.get(e.host)
|
||||
if (feed == null) {
|
||||
log.severe("Fetching non-existent feed " + e.host.getHumanReadableName())
|
||||
return
|
||||
}
|
||||
|
||||
feed.setStatus(e.status)
|
||||
|
||||
if (e.status.isActive())
|
||||
return
|
||||
|
||||
if (e.status == FeedFetchStatus.FINISHED) {
|
||||
feed.setStatus(FeedFetchStatus.IDLE)
|
||||
feed.setLastUpdated(e.getTimestamp())
|
||||
}
|
||||
// save feed items, then save feed. This will save partial fetches too
|
||||
// which is ok because the items are stored in a Set
|
||||
persister.submit({saveFeedItems(e.host)} as Runnable)
|
||||
persister.submit({saveFeedMetadata(feed)} as Runnable)
|
||||
}
|
||||
|
||||
void onUIFeedConfigurationEvent(UIFeedConfigurationEvent e) {
|
||||
feeds.put(e.feed.getPublisher(), e.feed)
|
||||
persister.submit({saveFeedMetadata(e.feed)} as Runnable)
|
||||
}
|
||||
|
||||
void onUIFeedDeletedEvent(UIFeedDeletedEvent e) {
|
||||
Feed f = feeds.get(e.host)
|
||||
if (f == null) {
|
||||
log.severe("Deleting a non-existing feed " + e.host.getHumanReadableName())
|
||||
return
|
||||
}
|
||||
persister.submit({deleteFeed(f)} as Runnable)
|
||||
}
|
||||
|
||||
private void saveFeedItems(Persona publisher) {
|
||||
Set<FeedItem> set = feedItems.get(publisher)
|
||||
if (set == null)
|
||||
return // can happen if nothing was published
|
||||
|
||||
Feed feed = feeds[publisher]
|
||||
if (feed == null) {
|
||||
log.severe("Persisting items for non-existing feed " + publisher.getHumanReadableName())
|
||||
return
|
||||
}
|
||||
|
||||
if (feed.getItemsToKeep() == 0)
|
||||
return
|
||||
|
||||
List<FeedItem> list = new ArrayList<>(set)
|
||||
if (feed.getItemsToKeep() > 0 && list.size() > feed.getItemsToKeep()) {
|
||||
log.info("will persist ${feed.getItemsToKeep()}/${list.size()} items")
|
||||
list.sort({l, r ->
|
||||
Long.compare(r.getTimestamp(), l.getTimestamp())
|
||||
} as Comparator<FeedItem>)
|
||||
list = list[0..feed.getItemsToKeep() - 1]
|
||||
}
|
||||
|
||||
|
||||
File itemsFile = getItemsFile(feed)
|
||||
itemsFile.withPrintWriter { writer ->
|
||||
list.each { item ->
|
||||
def obj = FeedItems.feedItemToObj(item)
|
||||
def json = JsonOutput.toJson(obj)
|
||||
writer.println(json)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void saveFeedMetadata(Feed feed) {
|
||||
File metadataFile = getMetadataFile(feed)
|
||||
metadataFile.withPrintWriter { writer ->
|
||||
def json = [:]
|
||||
json.publisher = feed.getPublisher().toBase64()
|
||||
json.itemsToKeep = feed.getItemsToKeep()
|
||||
json.lastUpdated = feed.getLastUpdated()
|
||||
json.updateInterval = feed.getUpdateInterval()
|
||||
json.autoDownload = feed.isAutoDownload()
|
||||
json.sequential = feed.isSequential()
|
||||
json.lastUpdateAttempt = feed.getLastUpdateAttempt()
|
||||
json = JsonOutput.toJson(json)
|
||||
writer.println(json)
|
||||
}
|
||||
}
|
||||
|
||||
private void deleteFeed(Feed feed) {
|
||||
feeds.remove(feed.getPublisher())
|
||||
feedItems.remove(feed.getPublisher())
|
||||
getItemsFile(feed).delete()
|
||||
getMetadataFile(feed).delete()
|
||||
}
|
||||
|
||||
private File getItemsFile(Feed feed) {
|
||||
return new File(itemsFolder, feed.getPublisher().destination.toBase32() + ".json")
|
||||
}
|
||||
|
||||
private File getMetadataFile(Feed feed) {
|
||||
return new File(metadataFolder, feed.getPublisher().destination.toBase32() + ".json")
|
||||
}
|
||||
}
|
||||
@ -1,9 +0,0 @@
|
||||
package com.muwire.core.filefeeds
|
||||
|
||||
import com.muwire.core.Event
|
||||
|
||||
class UIDownloadFeedItemEvent extends Event {
|
||||
FeedItem item
|
||||
File target
|
||||
boolean sequential
|
||||
}
|
||||
@ -1,12 +0,0 @@
|
||||
package com.muwire.core.filefeeds
|
||||
|
||||
import com.muwire.core.Event
|
||||
|
||||
/**
|
||||
* Emitted when configuration of a feed changes.
|
||||
* The object should already contain the updated values.
|
||||
*/
|
||||
class UIFeedConfigurationEvent extends Event {
|
||||
Feed feed
|
||||
boolean newFeed
|
||||
}
|
||||
@ -1,8 +0,0 @@
|
||||
package com.muwire.core.filefeeds
|
||||
|
||||
import com.muwire.core.Event
|
||||
import com.muwire.core.Persona
|
||||
|
||||
class UIFeedDeletedEvent extends Event {
|
||||
Persona host
|
||||
}
|
||||
@ -1,8 +0,0 @@
|
||||
package com.muwire.core.filefeeds
|
||||
|
||||
import com.muwire.core.Event
|
||||
import com.muwire.core.Persona
|
||||
|
||||
class UIFeedUpdateEvent extends Event {
|
||||
Persona host
|
||||
}
|
||||
@ -1,8 +0,0 @@
|
||||
package com.muwire.core.filefeeds
|
||||
|
||||
import com.muwire.core.Event
|
||||
import com.muwire.core.SharedFile
|
||||
|
||||
class UIFilePublishedEvent extends Event {
|
||||
SharedFile sf
|
||||
}
|
||||
@ -1,8 +0,0 @@
|
||||
package com.muwire.core.filefeeds
|
||||
|
||||
import com.muwire.core.Event
|
||||
import com.muwire.core.SharedFile
|
||||
|
||||
class UIFileUnpublishedEvent extends Event {
|
||||
SharedFile sf
|
||||
}
|
||||
@ -1,173 +0,0 @@
|
||||
package com.muwire.core.files
|
||||
|
||||
import com.muwire.core.DownloadedFile
|
||||
import com.muwire.core.InfoHash
|
||||
import com.muwire.core.Persona
|
||||
import com.muwire.core.Service
|
||||
import com.muwire.core.SharedFile
|
||||
import com.muwire.core.util.DataUtil
|
||||
import net.i2p.data.Base64
|
||||
import net.i2p.data.Destination
|
||||
|
||||
import java.util.stream.Collectors
|
||||
|
||||
abstract class BasePersisterService extends Service{
|
||||
|
||||
protected static FileLoadedEvent fromJson(def json) {
|
||||
if (json.file == null || json.length == null || json.infoHash == null || json.hashList == null)
|
||||
throw new IllegalArgumentException()
|
||||
if (!(json.hashList instanceof List))
|
||||
throw new IllegalArgumentException()
|
||||
|
||||
def file = new File(DataUtil.readi18nString(Base64.decode(json.file)))
|
||||
file = file.getCanonicalFile()
|
||||
if (!file.exists() || file.isDirectory())
|
||||
return null
|
||||
long length = Long.valueOf(json.length)
|
||||
if (length != file.length())
|
||||
return null
|
||||
|
||||
List hashList = (List) json.hashList
|
||||
ByteArrayOutputStream baos = new ByteArrayOutputStream()
|
||||
hashList.each {
|
||||
byte [] hash = Base64.decode it.toString()
|
||||
if (hash == null)
|
||||
throw new IllegalArgumentException()
|
||||
baos.write hash
|
||||
}
|
||||
byte[] hashListBytes = baos.toByteArray()
|
||||
|
||||
InfoHash ih = InfoHash.fromHashList(hashListBytes)
|
||||
byte [] root = Base64.decode(json.infoHash.toString())
|
||||
if (root == null)
|
||||
throw new IllegalArgumentException()
|
||||
if (!Arrays.equals(root, ih.getRoot()))
|
||||
return null
|
||||
|
||||
int pieceSize = 0
|
||||
if (json.pieceSize != null)
|
||||
pieceSize = json.pieceSize
|
||||
|
||||
if (json.sources != null) {
|
||||
List sources = (List)json.sources
|
||||
Set<Destination> sourceSet = sources.stream().map({ d -> new Destination(d.toString())}).collect Collectors.toSet()
|
||||
DownloadedFile df = new DownloadedFile(file, ih.getRoot(), pieceSize, sourceSet)
|
||||
df.setComment(json.comment)
|
||||
return new FileLoadedEvent(loadedFile : df, infoHash: ih)
|
||||
}
|
||||
|
||||
|
||||
SharedFile sf = new SharedFile(file, ih.getRoot(), pieceSize)
|
||||
sf.setComment(json.comment)
|
||||
if (json.downloaders != null)
|
||||
sf.getDownloaders().addAll(json.downloaders)
|
||||
if (json.searchers != null) {
|
||||
json.searchers.each {
|
||||
Persona searcher = null
|
||||
if (it.searcher != null)
|
||||
searcher = new Persona(new ByteArrayInputStream(Base64.decode(it.searcher)))
|
||||
long timestamp = it.timestamp
|
||||
String query = it.query
|
||||
sf.hit(searcher, timestamp, query)
|
||||
}
|
||||
}
|
||||
return new FileLoadedEvent(loadedFile: sf, infoHash: ih)
|
||||
|
||||
}
|
||||
|
||||
protected static FileLoadedEvent fromJsonLite(json) {
|
||||
if (json.file == null || json.length == null || json.root == null)
|
||||
throw new IllegalArgumentException()
|
||||
|
||||
def file = new File(DataUtil.readi18nString(Base64.decode(json.file)))
|
||||
file = file.getCanonicalFile()
|
||||
if (!file.exists() || file.isDirectory())
|
||||
return null
|
||||
long length = Long.valueOf(json.length)
|
||||
if (length != file.length())
|
||||
return null
|
||||
|
||||
byte[] root = Base64.decode(json.root)
|
||||
InfoHash ih = new InfoHash(root)
|
||||
|
||||
int pieceSize = 0
|
||||
if (json.pieceSize != null)
|
||||
pieceSize = json.pieceSize
|
||||
|
||||
boolean published = false
|
||||
long publishedTimestamp = -1
|
||||
if (json.published != null && json.published) {
|
||||
published = true
|
||||
publishedTimestamp = json.publishedTimestamp
|
||||
}
|
||||
|
||||
if (json.sources != null) {
|
||||
List sources = (List)json.sources
|
||||
Set<Destination> sourceSet = sources.stream().map({ d -> new Destination(d.toString())}).collect Collectors.toSet()
|
||||
DownloadedFile df = new DownloadedFile(file, ih.getRoot(), pieceSize, sourceSet)
|
||||
if (published)
|
||||
df.publish(publishedTimestamp)
|
||||
df.setComment(json.comment)
|
||||
return new FileLoadedEvent(loadedFile : df, infoHash: ih)
|
||||
}
|
||||
|
||||
|
||||
SharedFile sf = new SharedFile(file, ih.getRoot(), pieceSize)
|
||||
sf.setComment(json.comment)
|
||||
if (published)
|
||||
sf.publish(publishedTimestamp)
|
||||
if (json.downloaders != null)
|
||||
sf.getDownloaders().addAll(json.downloaders)
|
||||
if (json.searchers != null) {
|
||||
json.searchers.each {
|
||||
Persona searcher = null
|
||||
if (it.searcher != null) {
|
||||
try {
|
||||
searcher = new Persona(new ByteArrayInputStream(Base64.decode(it.searcher)))
|
||||
} catch (Exception ignore) {
|
||||
return
|
||||
}
|
||||
}
|
||||
long timestamp = it.timestamp
|
||||
String query = it.query
|
||||
sf.hit(searcher, timestamp, query)
|
||||
}
|
||||
}
|
||||
return new FileLoadedEvent(loadedFile: sf, infoHash: ih)
|
||||
}
|
||||
|
||||
protected static toJson(SharedFile sf) {
|
||||
def json = [:]
|
||||
json.file = sf.getB64EncodedFileName()
|
||||
json.length = sf.getCachedLength()
|
||||
json.root = Base64.encode(sf.getRoot())
|
||||
json.pieceSize = sf.getPieceSize()
|
||||
json.comment = sf.getComment()
|
||||
json.hits = sf.getHits()
|
||||
json.downloaders = sf.getDownloaders()
|
||||
|
||||
if (!sf.searches.isEmpty()) {
|
||||
Set searchers = new HashSet<>()
|
||||
sf.searches.each {
|
||||
def search = [:]
|
||||
if (it.searcher != null)
|
||||
search.searcher = it.searcher.toBase64()
|
||||
search.timestamp = it.timestamp
|
||||
search.query = it.query
|
||||
searchers.add(search)
|
||||
}
|
||||
json.searchers = searchers
|
||||
}
|
||||
|
||||
if (sf instanceof DownloadedFile) {
|
||||
json.sources = sf.sources.stream().map( {d -> d.toBase64()}).collect(Collectors.toList())
|
||||
}
|
||||
|
||||
if (sf.isPublished()) {
|
||||
json.published = true
|
||||
json.publishedTimestamp = sf.getPublishedTimestamp()
|
||||
}
|
||||
|
||||
json
|
||||
}
|
||||
}
|
||||
@ -4,9 +4,8 @@ import com.muwire.core.Event
|
||||
|
||||
class DirectoryUnsharedEvent extends Event {
|
||||
File directory
|
||||
boolean deleted
|
||||
|
||||
public String toString() {
|
||||
super.toString() + " unshared directory "+ directory.toString() + " deleted $deleted"
|
||||
super.toString() + " unshared directory "+ directory.toString()
|
||||
}
|
||||
}
|
||||
|
||||
@ -15,9 +15,6 @@ import java.util.concurrent.ConcurrentHashMap
|
||||
import com.muwire.core.EventBus
|
||||
import com.muwire.core.MuWireSettings
|
||||
import com.muwire.core.SharedFile
|
||||
import com.muwire.core.files.directories.WatchedDirectoryConfigurationEvent
|
||||
import com.muwire.core.files.directories.WatchedDirectoryConvertedEvent
|
||||
import com.muwire.core.files.directories.WatchedDirectoryManager
|
||||
|
||||
import groovy.util.logging.Log
|
||||
import net.i2p.util.SystemVersion
|
||||
@ -36,27 +33,27 @@ class DirectoryWatcher {
|
||||
}
|
||||
|
||||
private final File home
|
||||
private final MuWireSettings muOptions
|
||||
private final EventBus eventBus
|
||||
private final FileManager fileManager
|
||||
private final WatchedDirectoryManager watchedDirectoryManager
|
||||
private final Thread watcherThread, publisherThread
|
||||
private final Map<File, Long> waitingFiles = new ConcurrentHashMap<>()
|
||||
private final Map<File, WatchKey> watchedDirectories = new ConcurrentHashMap<>()
|
||||
private WatchService watchService
|
||||
private volatile boolean shutdown
|
||||
|
||||
DirectoryWatcher(EventBus eventBus, FileManager fileManager, File home, WatchedDirectoryManager watchedDirectoryManager) {
|
||||
DirectoryWatcher(EventBus eventBus, FileManager fileManager, File home, MuWireSettings muOptions) {
|
||||
this.home = home
|
||||
this.muOptions = muOptions
|
||||
this.eventBus = eventBus
|
||||
this.fileManager = fileManager
|
||||
this.watchedDirectoryManager = watchedDirectoryManager
|
||||
this.watcherThread = new Thread({watch() } as Runnable, "directory-watcher")
|
||||
watcherThread.setDaemon(true)
|
||||
this.publisherThread = new Thread({publish()} as Runnable, "watched-files-publisher")
|
||||
publisherThread.setDaemon(true)
|
||||
}
|
||||
|
||||
void onWatchedDirectoryConvertedEvent(WatchedDirectoryConvertedEvent e) {
|
||||
void onAllFilesLoadedEvent(AllFilesLoadedEvent e) {
|
||||
watchService = FileSystems.getDefault().newWatchService()
|
||||
watcherThread.start()
|
||||
publisherThread.start()
|
||||
@ -74,26 +71,26 @@ class DirectoryWatcher {
|
||||
Path path = canonical.toPath()
|
||||
WatchKey wk = path.register(watchService, kinds)
|
||||
watchedDirectories.put(canonical, wk)
|
||||
|
||||
if (muOptions.watchedDirectories.add(canonical.toString()))
|
||||
saveMuSettings()
|
||||
}
|
||||
|
||||
void onDirectoryUnsharedEvent(DirectoryUnsharedEvent e) {
|
||||
WatchKey wk = watchedDirectories.remove(e.directory)
|
||||
wk?.cancel()
|
||||
|
||||
if (muOptions.watchedDirectories.remove(e.directory.toString()))
|
||||
saveMuSettings()
|
||||
}
|
||||
|
||||
void onWatchedDirectoryConfigurationEvent(WatchedDirectoryConfigurationEvent e) {
|
||||
if (watchService == null)
|
||||
return // still converting
|
||||
if (!e.autoWatch) {
|
||||
WatchKey wk = watchedDirectories.remove(e.directory)
|
||||
wk?.cancel()
|
||||
} else if (!watchedDirectories.containsKey(e.directory)) {
|
||||
Path path = e.directory.toPath()
|
||||
def wk = path.register(watchService, kinds)
|
||||
watchedDirectories.put(e.directory, wk)
|
||||
} // else it was already watched
|
||||
private void saveMuSettings() {
|
||||
File muSettingsFile = new File(home, "MuWire.properties")
|
||||
muSettingsFile.withPrintWriter("UTF-8", {
|
||||
muOptions.write(it)
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
private void watch() {
|
||||
try {
|
||||
while(!shutdown) {
|
||||
@ -118,7 +115,7 @@ class DirectoryWatcher {
|
||||
File f= join(parent, path)
|
||||
log.fine("created entry $f")
|
||||
if (f.isDirectory())
|
||||
eventBus.publish(new FileSharedEvent(file : f, fromWatch : true))
|
||||
f.toPath().register(watchService, kinds)
|
||||
else
|
||||
waitingFiles.put(f, System.currentTimeMillis())
|
||||
}
|
||||
@ -136,10 +133,6 @@ class DirectoryWatcher {
|
||||
SharedFile sf = fileManager.fileToSharedFile.get(f)
|
||||
if (sf != null)
|
||||
eventBus.publish(new FileUnsharedEvent(unsharedFile : sf, deleted : true))
|
||||
else if (watchedDirectoryManager.isWatched(f))
|
||||
eventBus.publish(new DirectoryUnsharedEvent(directory : f, deleted : true))
|
||||
else
|
||||
log.fine("Entry was not relevant");
|
||||
}
|
||||
|
||||
private static File join(Path parent, Path path) {
|
||||
@ -156,7 +149,7 @@ class DirectoryWatcher {
|
||||
waitingFiles.each { file, timestamp ->
|
||||
if (now - timestamp > WAIT_TIME) {
|
||||
log.fine("publishing file $file")
|
||||
eventBus.publish new FileSharedEvent(file : file, fromWatch: true)
|
||||
eventBus.publish new FileSharedEvent(file : file)
|
||||
published << file
|
||||
}
|
||||
}
|
||||
|
||||
@ -2,7 +2,6 @@ package com.muwire.core.files
|
||||
|
||||
import com.muwire.core.DownloadedFile
|
||||
import com.muwire.core.Event
|
||||
import com.muwire.core.InfoHash
|
||||
import com.muwire.core.download.Downloader
|
||||
|
||||
import net.i2p.data.Destination
|
||||
@ -10,5 +9,4 @@ import net.i2p.data.Destination
|
||||
class FileDownloadedEvent extends Event {
|
||||
Downloader downloader
|
||||
DownloadedFile downloadedFile
|
||||
InfoHash infoHash
|
||||
}
|
||||
|
||||
@ -1,18 +1,16 @@
|
||||
package com.muwire.core.files
|
||||
|
||||
import com.muwire.core.Event
|
||||
import com.muwire.core.InfoHash
|
||||
import com.muwire.core.SharedFile
|
||||
|
||||
class FileHashedEvent extends Event {
|
||||
|
||||
SharedFile sharedFile
|
||||
InfoHash infoHash
|
||||
String error
|
||||
|
||||
@Override
|
||||
public String toString() {
|
||||
super.toString() + " sharedFile " + sharedFile?.file?.getAbsolutePath() + " error: $error"
|
||||
super.toString() + " sharedFile " + sharedFile?.file.getAbsolutePath() + " error: $error"
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@ -1,12 +1,9 @@
|
||||
package com.muwire.core.files
|
||||
|
||||
import com.muwire.core.Event
|
||||
import com.muwire.core.InfoHash
|
||||
import com.muwire.core.SharedFile
|
||||
|
||||
class FileLoadedEvent extends Event {
|
||||
|
||||
SharedFile loadedFile
|
||||
InfoHash infoHash
|
||||
String source
|
||||
}
|
||||
|
||||
@ -1,8 +1,5 @@
|
||||
package com.muwire.core.files
|
||||
|
||||
import java.util.stream.Collectors
|
||||
import java.util.stream.Stream
|
||||
|
||||
import com.muwire.core.EventBus
|
||||
import com.muwire.core.InfoHash
|
||||
import com.muwire.core.MuWireSettings
|
||||
@ -28,7 +25,6 @@ class FileManager {
|
||||
final Map<String, Set<File>> commentToFile = new HashMap<>()
|
||||
final SearchIndex index = new SearchIndex()
|
||||
final FileTree<Void> negativeTree = new FileTree<>()
|
||||
final FileTree<SharedFile> positiveTree = new FileTree<>()
|
||||
final Set<File> sideCarFiles = new HashSet<>()
|
||||
|
||||
FileManager(EventBus eventBus, MuWireSettings settings) {
|
||||
@ -79,7 +75,7 @@ class FileManager {
|
||||
|
||||
private void addToIndex(SharedFile sf) {
|
||||
log.info("Adding shared file " + sf.getFile())
|
||||
InfoHash infoHash = new InfoHash(sf.getRoot())
|
||||
InfoHash infoHash = sf.getInfoHash()
|
||||
Set<SharedFile> existing = rootToFiles.get(infoHash)
|
||||
if (existing == null) {
|
||||
log.info("adding new root")
|
||||
@ -88,7 +84,6 @@ class FileManager {
|
||||
}
|
||||
existing.add(sf)
|
||||
fileToSharedFile.put(sf.file, sf)
|
||||
positiveTree.add(sf.file, sf);
|
||||
|
||||
negativeTree.remove(sf.file)
|
||||
String parent = sf.getFile().getParent()
|
||||
@ -122,7 +117,7 @@ class FileManager {
|
||||
|
||||
void onFileUnsharedEvent(FileUnsharedEvent e) {
|
||||
SharedFile sf = e.unsharedFile
|
||||
InfoHash infoHash = new InfoHash(sf.getRoot())
|
||||
InfoHash infoHash = sf.getInfoHash()
|
||||
Set<SharedFile> existing = rootToFiles.get(infoHash)
|
||||
if (existing != null) {
|
||||
existing.remove(sf)
|
||||
@ -132,7 +127,6 @@ class FileManager {
|
||||
}
|
||||
|
||||
fileToSharedFile.remove(sf.file)
|
||||
positiveTree.remove(sf.file)
|
||||
if (!e.deleted && negativeTree.fileToNode.containsKey(sf.file.getParentFile())) {
|
||||
negativeTree.add(sf.file,null)
|
||||
saveNegativeTree()
|
||||
@ -196,10 +190,6 @@ class FileManager {
|
||||
Set<SharedFile> getSharedFiles(byte []root) {
|
||||
return rootToFiles.get(new InfoHash(root))
|
||||
}
|
||||
|
||||
boolean isShared(InfoHash infoHash) {
|
||||
rootToFiles.containsKey(infoHash)
|
||||
}
|
||||
|
||||
void onSearchEvent(SearchEvent e) {
|
||||
// hash takes precedence
|
||||
@ -249,26 +239,14 @@ class FileManager {
|
||||
void onDirectoryUnsharedEvent(DirectoryUnsharedEvent e) {
|
||||
negativeTree.remove(e.directory)
|
||||
saveNegativeTree()
|
||||
if (!e.deleted) {
|
||||
e.directory.listFiles().each {
|
||||
if (it.isDirectory())
|
||||
eventBus.publish(new DirectoryUnsharedEvent(directory : it))
|
||||
else {
|
||||
SharedFile sf = fileToSharedFile.get(it)
|
||||
if (sf != null)
|
||||
eventBus.publish(new FileUnsharedEvent(unsharedFile : sf))
|
||||
}
|
||||
e.directory.listFiles().each {
|
||||
if (it.isDirectory())
|
||||
eventBus.publish(new DirectoryUnsharedEvent(directory : it))
|
||||
else {
|
||||
SharedFile sf = fileToSharedFile.get(it)
|
||||
if (sf != null)
|
||||
eventBus.publish(new FileUnsharedEvent(unsharedFile : sf))
|
||||
}
|
||||
} else {
|
||||
def cb = new DirDeletionCallback()
|
||||
positiveTree.traverse(e.directory, cb)
|
||||
positiveTree.remove(e.directory)
|
||||
cb.unsharedFiles.each {
|
||||
eventBus.publish(new FileUnsharedEvent(unsharedFile : it, deleted: true))
|
||||
}
|
||||
cb.subDirs.each {
|
||||
eventBus.publish(new DirectoryUnsharedEvent(directory : it, deleted : true))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ -276,34 +254,4 @@ class FileManager {
|
||||
settings.negativeFileTree.clear()
|
||||
settings.negativeFileTree.addAll(negativeTree.fileToNode.keySet().collect { it.getAbsolutePath() })
|
||||
}
|
||||
|
||||
public List<SharedFile> getPublishedSince(long timestamp) {
|
||||
synchronized(fileToSharedFile) {
|
||||
fileToSharedFile.values().stream().
|
||||
filter({sf -> sf.isPublished()}).
|
||||
filter({sf -> sf.getPublishedTimestamp() >= timestamp}).
|
||||
collect(Collectors.toList())
|
||||
}
|
||||
}
|
||||
|
||||
private static class DirDeletionCallback implements FileTreeCallback<SharedFile> {
|
||||
|
||||
final List<File> subDirs = new ArrayList<>()
|
||||
final List<SharedFile> unsharedFiles = new ArrayList<>()
|
||||
|
||||
@Override
|
||||
public void onDirectoryEnter(File file) {
|
||||
subDirs.add(file)
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onDirectoryLeave() {
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onFile(File file, SharedFile value) {
|
||||
unsharedFiles << value
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
|
||||
@ -5,10 +5,9 @@ import com.muwire.core.Event
|
||||
class FileSharedEvent extends Event {
|
||||
|
||||
File file
|
||||
boolean fromWatch
|
||||
|
||||
@Override
|
||||
public String toString() {
|
||||
return super.toString() + " file: "+file.getAbsolutePath() + " fromWatch: $fromWatch"
|
||||
return super.toString() + " file: "+file.getAbsolutePath()
|
||||
}
|
||||
}
|
||||
|
||||
@ -23,7 +23,6 @@ class FileTree<T> {
|
||||
if (existing == null) {
|
||||
existing = new TreeNode()
|
||||
existing.file = element
|
||||
existing.isFile = element.isFile()
|
||||
existing.parent = current
|
||||
fileToNode.put(element, existing)
|
||||
current.children.add(existing)
|
||||
@ -65,7 +64,7 @@ class FileTree<T> {
|
||||
private void doTraverse(TreeNode<T> node, FileTreeCallback<T> callback) {
|
||||
boolean leave = false
|
||||
if (node.file != null) {
|
||||
if (node.isFile)
|
||||
if (node.file.isFile())
|
||||
callback.onFile(node.file, node.value)
|
||||
else {
|
||||
leave = true
|
||||
@ -89,7 +88,7 @@ class FileTree<T> {
|
||||
node = fileToNode.get(parent)
|
||||
|
||||
node.children.each {
|
||||
if (it.isFile)
|
||||
if (it.file.isFile())
|
||||
callback.onFile(it.file, it.value)
|
||||
else
|
||||
callback.onDirectory(it.file)
|
||||
@ -99,7 +98,6 @@ class FileTree<T> {
|
||||
public static class TreeNode<T> {
|
||||
TreeNode parent
|
||||
File file
|
||||
boolean isFile
|
||||
T value;
|
||||
final Set<TreeNode> children = new HashSet<>()
|
||||
|
||||
|
||||
@ -53,6 +53,7 @@ class HasherService {
|
||||
|
||||
private void process(File f) {
|
||||
if (f.isDirectory()) {
|
||||
eventBus.publish(new DirectoryWatchedEvent(directory : f))
|
||||
f.listFiles().each {
|
||||
eventBus.publish new FileSharedEvent(file: it)
|
||||
}
|
||||
@ -64,8 +65,7 @@ class HasherService {
|
||||
} else {
|
||||
eventBus.publish new FileHashingEvent(hashingFile: f)
|
||||
def hash = hasher.hashFile f
|
||||
eventBus.publish new FileHashedEvent(sharedFile: new SharedFile(f, hash.getRoot(), FileHasher.getPieceSize(f.length())),
|
||||
infoHash : hash)
|
||||
eventBus.publish new FileHashedEvent(sharedFile: new SharedFile(f, hash, FileHasher.getPieceSize(f.length())))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ -1,12 +0,0 @@
|
||||
package com.muwire.core.files
|
||||
|
||||
import com.muwire.core.Event
|
||||
|
||||
/**
|
||||
* Should be triggered by the old PersisterService
|
||||
* once it has finished reading the old file
|
||||
*
|
||||
* @see PersisterService
|
||||
*/
|
||||
class PersisterDoneEvent extends Event{
|
||||
}
|
||||
@ -1,195 +0,0 @@
|
||||
package com.muwire.core.files
|
||||
|
||||
import com.muwire.core.*
|
||||
import com.muwire.core.filefeeds.UIFilePublishedEvent
|
||||
import com.muwire.core.filefeeds.UIFileUnpublishedEvent
|
||||
|
||||
import groovy.json.JsonOutput
|
||||
import groovy.json.JsonSlurper
|
||||
import groovy.util.logging.Log
|
||||
|
||||
import java.nio.file.Files
|
||||
import java.nio.file.Path
|
||||
import java.nio.file.Paths
|
||||
import java.util.concurrent.ExecutorService
|
||||
import java.util.concurrent.Executors
|
||||
import java.util.concurrent.ThreadFactory
|
||||
import java.util.logging.Level
|
||||
|
||||
/**
|
||||
* A persister that stores information about the files shared using
|
||||
* individual JSON files in directories.
|
||||
*
|
||||
* The absolute path's 32bit hash to the shared file is used
|
||||
* to build the directory and filename.
|
||||
*
|
||||
* This persister only starts working once the old persister has finished loading
|
||||
* @see PersisterFolderService#getJsonPath
|
||||
*/
|
||||
@Log
|
||||
class PersisterFolderService extends BasePersisterService {
|
||||
|
||||
final static int CUT_LENGTH = 6
|
||||
|
||||
private final Core core;
|
||||
final File location
|
||||
final EventBus listener
|
||||
final int interval
|
||||
final Timer timer
|
||||
final ExecutorService persisterExecutor = Executors.newSingleThreadExecutor({ r ->
|
||||
new Thread(r, "file persister")
|
||||
} as ThreadFactory)
|
||||
|
||||
PersisterFolderService(Core core, File location, EventBus listener) {
|
||||
this.core = core;
|
||||
this.location = location
|
||||
this.listener = listener
|
||||
this.interval = interval
|
||||
timer = new Timer("file-folder persister timer", true)
|
||||
}
|
||||
|
||||
void stop() {
|
||||
timer.cancel()
|
||||
persisterExecutor.shutdown()
|
||||
}
|
||||
|
||||
void onPersisterDoneEvent(PersisterDoneEvent persisterDoneEvent) {
|
||||
log.info("Old persister done")
|
||||
load()
|
||||
}
|
||||
|
||||
void onFileHashedEvent(FileHashedEvent hashedEvent) {
|
||||
if (core.getMuOptions().getAutoPublishSharedFiles() && hashedEvent.sharedFile != null)
|
||||
hashedEvent.sharedFile.publish(System.currentTimeMillis())
|
||||
persistFile(hashedEvent.sharedFile, hashedEvent.infoHash)
|
||||
}
|
||||
|
||||
void onFileDownloadedEvent(FileDownloadedEvent downloadedEvent) {
|
||||
if (core.getMuOptions().getShareDownloadedFiles()) {
|
||||
if (core.getMuOptions().getAutoPublishSharedFiles())
|
||||
downloadedEvent.downloadedFile.publish(System.currentTimeMillis())
|
||||
persistFile(downloadedEvent.downloadedFile, downloadedEvent.infoHash)
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Get rid of the json and hashlists of unshared files
|
||||
* @param unsharedEvent
|
||||
*/
|
||||
void onFileUnsharedEvent(FileUnsharedEvent unsharedEvent) {
|
||||
def jsonPath = getJsonPath(unsharedEvent.unsharedFile)
|
||||
def jsonFile = jsonPath.toFile()
|
||||
if(jsonFile.isFile()){
|
||||
jsonFile.delete()
|
||||
}
|
||||
def hashListPath = getHashListPath(unsharedEvent.unsharedFile)
|
||||
def hashListFile = hashListPath.toFile()
|
||||
if (hashListFile.isFile())
|
||||
hashListFile.delete()
|
||||
}
|
||||
|
||||
void onFileLoadedEvent(FileLoadedEvent loadedEvent) {
|
||||
if(loadedEvent.source == "PersisterService"){
|
||||
log.info("Migrating persisted file from PersisterService: "
|
||||
+ loadedEvent.loadedFile.file.absolutePath.toString())
|
||||
persistFile(loadedEvent.loadedFile, loadedEvent.infoHash)
|
||||
}
|
||||
}
|
||||
|
||||
void onUICommentEvent(UICommentEvent e) {
|
||||
persistFile(e.sharedFile,null)
|
||||
}
|
||||
|
||||
void onUIFilePublishedEvent(UIFilePublishedEvent e) {
|
||||
persistFile(e.sf, null)
|
||||
}
|
||||
|
||||
void onUIFileUnpublishedEvent(UIFileUnpublishedEvent e) {
|
||||
persistFile(e.sf, null)
|
||||
}
|
||||
|
||||
void load() {
|
||||
log.fine("Loading...")
|
||||
Thread.currentThread().setPriority(Thread.MIN_PRIORITY)
|
||||
|
||||
if (location.exists() && location.isDirectory()) {
|
||||
try {
|
||||
_load()
|
||||
}
|
||||
catch (Exception e) {
|
||||
log.log(Level.WARNING, "couldn't load files", e)
|
||||
}
|
||||
} else {
|
||||
location.mkdirs()
|
||||
listener.publish(new AllFilesLoadedEvent())
|
||||
}
|
||||
loaded = true
|
||||
}
|
||||
|
||||
/**
|
||||
* Loads every JSON into memory
|
||||
*/
|
||||
private void _load() {
|
||||
int loaded = 0
|
||||
def slurper = new JsonSlurper()
|
||||
Files.walk(location.toPath())
|
||||
.filter({
|
||||
it.getFileName().toString().endsWith(".json")
|
||||
})
|
||||
.forEach({
|
||||
def parsed = slurper.parse it.toFile()
|
||||
def event = fromJsonLite parsed
|
||||
if (event == null) return
|
||||
|
||||
log.fine("loaded file $event.loadedFile.file")
|
||||
listener.publish event
|
||||
loaded++
|
||||
if (loaded % 10 == 0)
|
||||
Thread.sleep(20)
|
||||
|
||||
})
|
||||
listener.publish(new AllFilesLoadedEvent())
|
||||
}
|
||||
|
||||
private void persistFile(SharedFile sf, InfoHash ih) {
|
||||
persisterExecutor.submit({
|
||||
def jsonPath = getJsonPath(sf)
|
||||
|
||||
def startTime = System.currentTimeMillis()
|
||||
jsonPath.parent.toFile().mkdirs()
|
||||
jsonPath.toFile().withPrintWriter { writer ->
|
||||
def json = toJson sf
|
||||
json = JsonOutput.toJson(json)
|
||||
writer.println json
|
||||
}
|
||||
|
||||
if (ih != null) {
|
||||
def hashListPath = getHashListPath(sf)
|
||||
hashListPath.toFile().bytes = ih.hashList
|
||||
}
|
||||
log.fine("Time(ms) to write json+hashList: " + (System.currentTimeMillis() - startTime))
|
||||
} as Runnable)
|
||||
}
|
||||
private Path getJsonPath(SharedFile sf){
|
||||
def pathHash = sf.getB64PathHash()
|
||||
return Paths.get(
|
||||
location.getAbsolutePath(),
|
||||
pathHash.substring(0, CUT_LENGTH),
|
||||
pathHash.substring(CUT_LENGTH) + ".json"
|
||||
)
|
||||
}
|
||||
|
||||
private Path getHashListPath(SharedFile sf) {
|
||||
def pathHash = sf.getB64PathHash()
|
||||
return Paths.get(
|
||||
location.getAbsolutePath(),
|
||||
pathHash.substring(0, CUT_LENGTH),
|
||||
pathHash.substring(CUT_LENGTH) + ".hashlist"
|
||||
)
|
||||
}
|
||||
|
||||
InfoHash loadInfoHash(SharedFile sf) {
|
||||
def path = getHashListPath(sf)
|
||||
InfoHash.fromHashList(path.toFile().bytes)
|
||||
}
|
||||
}
|
||||
@ -1,24 +1,40 @@
|
||||
package com.muwire.core.files
|
||||
|
||||
|
||||
import java.nio.file.CopyOption
|
||||
import java.nio.file.Files
|
||||
import java.nio.file.StandardCopyOption
|
||||
import java.util.concurrent.ExecutorService
|
||||
import java.util.concurrent.Executors
|
||||
import java.util.concurrent.ThreadFactory
|
||||
import java.util.logging.Level
|
||||
import java.util.stream.Collectors
|
||||
|
||||
import com.muwire.core.DownloadedFile
|
||||
import com.muwire.core.EventBus
|
||||
import com.muwire.core.InfoHash
|
||||
import com.muwire.core.Persona
|
||||
import com.muwire.core.Service
|
||||
import com.muwire.core.SharedFile
|
||||
import com.muwire.core.UILoadedEvent
|
||||
import com.muwire.core.util.DataUtil
|
||||
|
||||
import groovy.json.JsonOutput
|
||||
import groovy.json.JsonSlurper
|
||||
import groovy.util.logging.Log
|
||||
import net.i2p.data.Base64
|
||||
import net.i2p.data.Destination
|
||||
|
||||
@Log
|
||||
class PersisterService extends BasePersisterService {
|
||||
class PersisterService extends Service {
|
||||
|
||||
final File location
|
||||
final EventBus listener
|
||||
final int interval
|
||||
final Timer timer
|
||||
final FileManager fileManager
|
||||
final ExecutorService persisterExecutor = Executors.newSingleThreadExecutor({ r ->
|
||||
new Thread(r, "file persister")
|
||||
} as ThreadFactory)
|
||||
|
||||
PersisterService(File location, EventBus listener, int interval, FileManager fileManager) {
|
||||
this.location = location
|
||||
@ -35,6 +51,10 @@ class PersisterService extends BasePersisterService {
|
||||
void onUILoadedEvent(UILoadedEvent e) {
|
||||
timer.schedule({load()} as TimerTask, 1)
|
||||
}
|
||||
|
||||
void onUIPersistFilesEvent(UIPersistFilesEvent e) {
|
||||
persistFiles()
|
||||
}
|
||||
|
||||
void load() {
|
||||
Thread.currentThread().setPriority(Thread.MIN_PRIORITY)
|
||||
@ -49,7 +69,6 @@ class PersisterService extends BasePersisterService {
|
||||
def event = fromJson parsed
|
||||
if (event != null) {
|
||||
log.fine("loaded file $event.loadedFile.file")
|
||||
event.source = "PersisterService"
|
||||
listener.publish event
|
||||
loaded++
|
||||
if (loaded % 10 == 0)
|
||||
@ -57,18 +76,126 @@ class PersisterService extends BasePersisterService {
|
||||
}
|
||||
}
|
||||
}
|
||||
// Backup the old hashes
|
||||
location.renameTo(
|
||||
new File(location.absolutePath + ".bak")
|
||||
)
|
||||
listener.publish(new PersisterDoneEvent())
|
||||
} catch (Exception e) {
|
||||
listener.publish(new AllFilesLoadedEvent())
|
||||
} catch (IllegalArgumentException|NumberFormatException e) {
|
||||
log.log(Level.WARNING, "couldn't load files",e)
|
||||
}
|
||||
} else {
|
||||
listener.publish(new PersisterDoneEvent())
|
||||
listener.publish(new AllFilesLoadedEvent())
|
||||
}
|
||||
timer.schedule({persistFiles()} as TimerTask, 1000, interval)
|
||||
loaded = true
|
||||
}
|
||||
|
||||
private static FileLoadedEvent fromJson(def json) {
|
||||
if (json.file == null || json.length == null || json.infoHash == null || json.hashList == null)
|
||||
throw new IllegalArgumentException()
|
||||
if (!(json.hashList instanceof List))
|
||||
throw new IllegalArgumentException()
|
||||
|
||||
def file = new File(DataUtil.readi18nString(Base64.decode(json.file)))
|
||||
file = file.getCanonicalFile()
|
||||
if (!file.exists() || file.isDirectory())
|
||||
return null
|
||||
long length = Long.valueOf(json.length)
|
||||
if (length != file.length())
|
||||
return null
|
||||
|
||||
List hashList = (List) json.hashList
|
||||
ByteArrayOutputStream baos = new ByteArrayOutputStream()
|
||||
hashList.each {
|
||||
byte [] hash = Base64.decode it.toString()
|
||||
if (hash == null)
|
||||
throw new IllegalArgumentException()
|
||||
baos.write hash
|
||||
}
|
||||
byte[] hashListBytes = baos.toByteArray()
|
||||
|
||||
InfoHash ih = InfoHash.fromHashList(hashListBytes)
|
||||
byte [] root = Base64.decode(json.infoHash.toString())
|
||||
if (root == null)
|
||||
throw new IllegalArgumentException()
|
||||
if (!Arrays.equals(root, ih.getRoot()))
|
||||
return null
|
||||
|
||||
int pieceSize = 0
|
||||
if (json.pieceSize != null)
|
||||
pieceSize = json.pieceSize
|
||||
|
||||
if (json.sources != null) {
|
||||
List sources = (List)json.sources
|
||||
Set<Destination> sourceSet = sources.stream().map({d -> new Destination(d.toString())}).collect Collectors.toSet()
|
||||
DownloadedFile df = new DownloadedFile(file, ih, pieceSize, sourceSet)
|
||||
df.setComment(json.comment)
|
||||
return new FileLoadedEvent(loadedFile : df)
|
||||
}
|
||||
|
||||
|
||||
SharedFile sf = new SharedFile(file, ih, pieceSize)
|
||||
sf.setComment(json.comment)
|
||||
if (json.downloaders != null)
|
||||
sf.getDownloaders().addAll(json.downloaders)
|
||||
if (json.searchers != null) {
|
||||
json.searchers.each {
|
||||
Persona searcher = null
|
||||
if (it.searcher != null)
|
||||
searcher = new Persona(new ByteArrayInputStream(Base64.decode(it.searcher)))
|
||||
long timestamp = it.timestamp
|
||||
String query = it.query
|
||||
sf.hit(searcher, timestamp, query)
|
||||
}
|
||||
}
|
||||
return new FileLoadedEvent(loadedFile: sf)
|
||||
|
||||
}
|
||||
|
||||
private void persistFiles() {
|
||||
persisterExecutor.submit( {
|
||||
def sharedFiles = fileManager.getSharedFiles()
|
||||
|
||||
File tmp = File.createTempFile("muwire-files", "tmp")
|
||||
tmp.deleteOnExit()
|
||||
tmp.withPrintWriter { writer ->
|
||||
sharedFiles.each { k, v ->
|
||||
def json = toJson(k,v)
|
||||
json = JsonOutput.toJson(json)
|
||||
writer.println json
|
||||
}
|
||||
}
|
||||
Files.copy(tmp.toPath(), location.toPath(), StandardCopyOption.REPLACE_EXISTING)
|
||||
tmp.delete()
|
||||
} as Runnable)
|
||||
}
|
||||
|
||||
private def toJson(File f, SharedFile sf) {
|
||||
def json = [:]
|
||||
json.file = sf.getB64EncodedFileName()
|
||||
json.length = sf.getCachedLength()
|
||||
InfoHash ih = sf.getInfoHash()
|
||||
json.infoHash = sf.getB64EncodedHashRoot()
|
||||
json.pieceSize = sf.getPieceSize()
|
||||
json.hashList = sf.getB64EncodedHashList()
|
||||
json.comment = sf.getComment()
|
||||
json.hits = sf.getHits()
|
||||
json.downloaders = sf.getDownloaders()
|
||||
|
||||
if (!sf.searches.isEmpty()) {
|
||||
Set searchers = new HashSet<>()
|
||||
sf.searches.each {
|
||||
def search = [:]
|
||||
if (it.searcher != null)
|
||||
search.searcher = it.searcher.toBase64()
|
||||
search.timestamp = it.timestamp
|
||||
search.query = it.query
|
||||
searchers.add(search)
|
||||
}
|
||||
json.searchers = searchers
|
||||
}
|
||||
|
||||
if (sf instanceof DownloadedFile) {
|
||||
json.sources = sf.sources.stream().map( {d -> d.toBase64()}).collect(Collectors.toList())
|
||||
}
|
||||
|
||||
json
|
||||
}
|
||||
}
|
||||
|
||||
@ -0,0 +1,6 @@
|
||||
package com.muwire.core.files
|
||||
|
||||
import com.muwire.core.Event
|
||||
|
||||
class UIPersistFilesEvent extends Event {
|
||||
}
|
||||
@ -1,7 +0,0 @@
|
||||
package com.muwire.core.files.directories
|
||||
|
||||
import com.muwire.core.Event
|
||||
|
||||
class UISyncDirectoryEvent extends Event {
|
||||
File directory
|
||||
}
|
||||
@ -1,37 +0,0 @@
|
||||
package com.muwire.core.files.directories
|
||||
|
||||
import com.muwire.core.util.DataUtil
|
||||
|
||||
import net.i2p.data.Base64
|
||||
|
||||
class WatchedDirectory {
|
||||
final File directory
|
||||
final String encodedName
|
||||
boolean autoWatch
|
||||
int syncInterval
|
||||
long lastSync
|
||||
|
||||
WatchedDirectory(File directory) {
|
||||
this.directory = directory.getCanonicalFile()
|
||||
this.encodedName = Base64.encode(DataUtil.encodei18nString(directory.getAbsolutePath()))
|
||||
}
|
||||
|
||||
def toJson() {
|
||||
def rv = [:]
|
||||
rv.directory = encodedName
|
||||
rv.autoWatch = autoWatch
|
||||
rv.syncInterval = syncInterval
|
||||
rv.lastSync = lastSync
|
||||
rv
|
||||
}
|
||||
|
||||
static WatchedDirectory fromJson(def json) {
|
||||
String dirName = DataUtil.readi18nString(Base64.decode(json.directory))
|
||||
File dir = new File(dirName)
|
||||
def rv = new WatchedDirectory(dir)
|
||||
rv.autoWatch = json.autoWatch
|
||||
rv.syncInterval = json.syncInterval
|
||||
rv.lastSync = json.lastSync
|
||||
rv
|
||||
}
|
||||
}
|
||||
@ -1,9 +0,0 @@
|
||||
package com.muwire.core.files.directories
|
||||
|
||||
import com.muwire.core.Event
|
||||
|
||||
class WatchedDirectoryConfigurationEvent extends Event {
|
||||
File directory
|
||||
boolean autoWatch
|
||||
int syncInterval
|
||||
}
|
||||
@ -1,10 +0,0 @@
|
||||
package com.muwire.core.files.directories
|
||||
|
||||
import com.muwire.core.Event
|
||||
|
||||
/**
|
||||
* Emitted when converting an old watched directory entry to the
|
||||
* new format.
|
||||
*/
|
||||
class WatchedDirectoryConvertedEvent extends Event {
|
||||
}
|
||||
@ -1,27 +0,0 @@
|
||||
package com.muwire.core.files.directories
|
||||
|
||||
import com.muwire.core.Core
|
||||
import com.muwire.core.files.AllFilesLoadedEvent
|
||||
|
||||
/**
|
||||
* converts the setting-based format to new folder-based format.
|
||||
*/
|
||||
class WatchedDirectoryConverter {
|
||||
|
||||
private final Core core
|
||||
|
||||
WatchedDirectoryConverter(Core core) {
|
||||
this.core = core
|
||||
}
|
||||
|
||||
void onAllFilesLoadedEvent(AllFilesLoadedEvent e) {
|
||||
core.getMuOptions().getWatchedDirectories().each {
|
||||
File directory = new File(it)
|
||||
directory = directory.getCanonicalFile()
|
||||
core.eventBus.publish(new WatchedDirectoryConfigurationEvent(directory : directory, autoWatch: true))
|
||||
}
|
||||
core.getMuOptions().getWatchedDirectories().clear()
|
||||
core.saveMuSettings()
|
||||
core.eventBus.publish(new WatchedDirectoryConvertedEvent())
|
||||
}
|
||||
}
|
||||
@ -1,224 +0,0 @@
|
||||
package com.muwire.core.files.directories
|
||||
|
||||
import java.nio.file.Files
|
||||
import java.util.concurrent.ConcurrentHashMap
|
||||
import java.util.concurrent.ExecutorService
|
||||
import java.util.concurrent.Executors
|
||||
import java.util.concurrent.ThreadFactory
|
||||
import java.util.stream.Stream
|
||||
|
||||
import com.muwire.core.EventBus
|
||||
import com.muwire.core.SharedFile
|
||||
import com.muwire.core.files.DirectoryUnsharedEvent
|
||||
import com.muwire.core.files.DirectoryWatchedEvent
|
||||
import com.muwire.core.files.FileListCallback
|
||||
import com.muwire.core.files.FileManager
|
||||
import com.muwire.core.files.FileSharedEvent
|
||||
import com.muwire.core.files.FileUnsharedEvent
|
||||
|
||||
import groovy.json.JsonOutput
|
||||
import groovy.json.JsonSlurper
|
||||
import groovy.util.logging.Log
|
||||
|
||||
@Log
|
||||
class WatchedDirectoryManager {
|
||||
|
||||
private final File home
|
||||
private final EventBus eventBus
|
||||
private final FileManager fileManager
|
||||
|
||||
private final Map<File, WatchedDirectory> watchedDirs = new ConcurrentHashMap<>()
|
||||
|
||||
private final ExecutorService diskIO = Executors.newSingleThreadExecutor({r ->
|
||||
Thread t = new Thread(r, "disk-io")
|
||||
t.setDaemon(true)
|
||||
t
|
||||
} as ThreadFactory)
|
||||
|
||||
private final Timer timer = new Timer("directory-timer", true)
|
||||
|
||||
private boolean converting = true
|
||||
|
||||
WatchedDirectoryManager(File home, EventBus eventBus, FileManager fileManager) {
|
||||
this.home = new File(home, "directories")
|
||||
this.home.mkdir()
|
||||
this.eventBus = eventBus
|
||||
this.fileManager = fileManager
|
||||
}
|
||||
|
||||
public boolean isWatched(File f) {
|
||||
watchedDirs.containsKey(f)
|
||||
}
|
||||
|
||||
public Stream<WatchedDirectory> getWatchedDirsStream() {
|
||||
watchedDirs.values().stream()
|
||||
}
|
||||
|
||||
public void shutdown() {
|
||||
diskIO.shutdown()
|
||||
timer.cancel()
|
||||
}
|
||||
|
||||
void onUISyncDirectoryEvent(UISyncDirectoryEvent e) {
|
||||
def wd = watchedDirs.get(e.directory)
|
||||
if (wd == null) {
|
||||
log.warning("Got a sync event for non-watched dir ${e.directory}")
|
||||
return
|
||||
}
|
||||
diskIO.submit({sync(wd, System.currentTimeMillis())} as Runnable)
|
||||
}
|
||||
|
||||
void onWatchedDirectoryConfigurationEvent(WatchedDirectoryConfigurationEvent e) {
|
||||
if (converting) {
|
||||
def newDir = new WatchedDirectory(e.directory)
|
||||
// conversion is always autowatch really
|
||||
newDir.autoWatch = e.autoWatch
|
||||
persist(newDir)
|
||||
} else {
|
||||
def wd = watchedDirs.get(e.directory)
|
||||
if (wd == null) {
|
||||
log.severe("got a configuration event for a non-watched directory ${e.directory}")
|
||||
return
|
||||
}
|
||||
wd.autoWatch = e.autoWatch
|
||||
wd.syncInterval = e.syncInterval
|
||||
persist(wd)
|
||||
}
|
||||
}
|
||||
|
||||
void onWatchedDirectoryConvertedEvent(WatchedDirectoryConvertedEvent e) {
|
||||
converting = false
|
||||
diskIO.submit({
|
||||
def slurper = new JsonSlurper()
|
||||
Files.walk(home.toPath()).filter({
|
||||
it.getFileName().toString().endsWith(".json")
|
||||
}).
|
||||
forEach {
|
||||
def parsed = slurper.parse(it.toFile())
|
||||
WatchedDirectory wd = WatchedDirectory.fromJson(parsed)
|
||||
if (wd.directory.exists() && wd.directory.isDirectory()) // check if directory disappeared
|
||||
watchedDirs.put(wd.directory, wd)
|
||||
else
|
||||
it.toFile().delete()
|
||||
}
|
||||
|
||||
watchedDirs.values().stream().filter({it.autoWatch}).forEach {
|
||||
eventBus.publish(new DirectoryWatchedEvent(directory : it.directory))
|
||||
eventBus.publish(new FileSharedEvent(file : it.directory))
|
||||
}
|
||||
timer.schedule({sync()} as TimerTask, 1000, 1000)
|
||||
} as Runnable)
|
||||
}
|
||||
|
||||
private void persist(WatchedDirectory dir) {
|
||||
diskIO.submit({doPersist(dir)} as Runnable)
|
||||
}
|
||||
|
||||
private void doPersist(WatchedDirectory dir) {
|
||||
def json = JsonOutput.toJson(dir.toJson())
|
||||
def targetFile = new File(home, dir.getEncodedName() + ".json")
|
||||
targetFile.text = json
|
||||
}
|
||||
|
||||
void onFileSharedEvent(FileSharedEvent e) {
|
||||
if (e.file.isFile() || watchedDirs.containsKey(e.file))
|
||||
return
|
||||
|
||||
def wd = new WatchedDirectory(e.file)
|
||||
if (e.fromWatch) {
|
||||
// parent should be already watched, copy settings
|
||||
def parent = watchedDirs.get(e.file.getParentFile())
|
||||
if (parent == null) {
|
||||
log.severe("watching found a directory without a watched parent? ${e.file}")
|
||||
return
|
||||
}
|
||||
wd.autoWatch = parent.autoWatch
|
||||
wd.syncInterval = parent.syncInterval
|
||||
} else
|
||||
wd.autoWatch = true
|
||||
|
||||
watchedDirs.put(wd.directory, wd)
|
||||
persist(wd)
|
||||
if (wd.autoWatch)
|
||||
eventBus.publish(new DirectoryWatchedEvent(directory: wd.directory))
|
||||
}
|
||||
|
||||
void onDirectoryUnsharedEvent(DirectoryUnsharedEvent e) {
|
||||
def wd = watchedDirs.remove(e.directory)
|
||||
if (wd == null) {
|
||||
log.warning("unshared a directory that wasn't watched? ${e.directory}")
|
||||
return
|
||||
}
|
||||
|
||||
File persistFile = new File(home, wd.getEncodedName() + ".json")
|
||||
persistFile.delete()
|
||||
}
|
||||
|
||||
private void sync() {
|
||||
long now = System.currentTimeMillis()
|
||||
watchedDirs.values().stream().
|
||||
filter({!it.autoWatch}).
|
||||
filter({it.syncInterval > 0}).
|
||||
filter({it.lastSync + it.syncInterval * 1000 < now}).
|
||||
forEach({wd -> diskIO.submit({sync(wd, now)} as Runnable )})
|
||||
}
|
||||
|
||||
private void sync(WatchedDirectory wd, long now) {
|
||||
log.fine("syncing ${wd.directory}")
|
||||
wd.lastSync = now
|
||||
doPersist(wd)
|
||||
eventBus.publish(new WatchedDirectorySyncEvent(directory: wd.directory, when: now))
|
||||
|
||||
def cb = new DirSyncCallback()
|
||||
fileManager.positiveTree.list(wd.directory, cb)
|
||||
|
||||
Set<File> filesOnFS = new HashSet<>()
|
||||
Set<File> dirsOnFS = new HashSet<>()
|
||||
wd.directory.listFiles().each {
|
||||
File canonical = it.getCanonicalFile()
|
||||
if (canonical.isFile())
|
||||
filesOnFS.add(canonical)
|
||||
else
|
||||
dirsOnFS.add(canonical)
|
||||
}
|
||||
|
||||
Set<File> addedFiles = new HashSet<>(filesOnFS)
|
||||
addedFiles.removeAll(cb.files)
|
||||
addedFiles.each {
|
||||
eventBus.publish(new FileSharedEvent(file : it, fromWatch : true))
|
||||
}
|
||||
Set<File> addedDirs = new HashSet<>(dirsOnFS)
|
||||
addedDirs.removeAll(cb.dirs)
|
||||
addedDirs.each {
|
||||
eventBus.publish(new FileSharedEvent(file : it, fromWatch : true))
|
||||
}
|
||||
|
||||
Set<File> deletedFiles = new HashSet<>(cb.files)
|
||||
deletedFiles.removeAll(filesOnFS)
|
||||
deletedFiles.each {
|
||||
eventBus.publish(new FileUnsharedEvent(unsharedFile : fileManager.getFileToSharedFile().get(it), deleted : true))
|
||||
}
|
||||
Set<File> deletedDirs = new HashSet<>(cb.dirs)
|
||||
deletedDirs.removeAll(dirsOnFS)
|
||||
deletedDirs.each {
|
||||
eventBus.publish(new DirectoryUnsharedEvent(directory : it, deleted: true))
|
||||
}
|
||||
}
|
||||
|
||||
private static class DirSyncCallback implements FileListCallback<SharedFile> {
|
||||
|
||||
private final Set<File> files = new HashSet<>()
|
||||
private final Set<File> dirs = new HashSet<>()
|
||||
|
||||
@Override
|
||||
public void onFile(File f, SharedFile value) {
|
||||
files.add(f)
|
||||
}
|
||||
|
||||
@Override
|
||||
public void onDirectory(File f) {
|
||||
dirs.add(f)
|
||||
}
|
||||
|
||||
}
|
||||
}
|
||||
@ -1,8 +0,0 @@
|
||||
package com.muwire.core.files.directories
|
||||
|
||||
import com.muwire.core.Event
|
||||
|
||||
class WatchedDirectorySyncEvent extends Event {
|
||||
File directory
|
||||
long when
|
||||
}
|
||||
@ -127,8 +127,7 @@ class CacheClient {
|
||||
|
||||
@Override
|
||||
public void disconnected(I2PSession session) {
|
||||
if (!stopped.get())
|
||||
log.severe "Cache client I2P session disconnected"
|
||||
log.severe "I2P session disconnected"
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -8,8 +8,10 @@ class CacheServers {
|
||||
private static Set<Destination> CACHES = [
|
||||
// zlatinb
|
||||
new Destination("Wddh2E6FyyXBF7SvUYHKdN-vjf3~N6uqQWNeBDTM0P33YjiQCOsyedrjmDZmWFrXUJfJLWnCb5bnKezfk4uDaMyj~uvDG~yvLVcFgcPWSUd7BfGgym-zqcG1q1DcM8vfun-US7YamBlmtC6MZ2j-~Igqzmgshita8aLPCfNAA6S6e2UMjjtG7QIXlxpMec75dkHdJlVWbzrk9z8Qgru3YIk0UztYgEwDNBbm9wInsbHhr3HtAfa02QcgRVqRN2PnQXuqUJs7R7~09FZPEviiIcUpkY3FeyLlX1sgQFBeGeA96blaPvZNGd6KnNdgfLgMebx5SSxC-N4KZMSMBz5cgonQF3~m2HHFRSI85zqZNG5X9bJN85t80ltiv1W1es8ZnQW4es11r7MrvJNXz5bmSH641yJIvS6qI8OJJNpFVBIQSXLD-96TayrLQPaYw~uNZ-eXaE6G5dYhiuN8xHsFI1QkdaUaVZnvDGfsRbpS5GtpUbBDbyLkdPurG0i7dN1wAAAA"),
|
||||
// echelon
|
||||
new Destination("2MJTl8gYVPK43iJZJa~-5K1OchgPaPHXpqZmKIiKFvxyy8BlIJzUSrF4mazdta--shFHISfT0PEeI95j1yDyKMpGxatUyjSt3ZnyTfAehQR-H2kYV9FvjHo68uA9X5AaGYHKRYLuWMkihMXygd8ywoLjZtFP0UbKMPggfOZaWmjHF4081XoUXt~7MEAeYSQowndiUx0AH3HxNEiv0N373JJS61OsIXb5ctqVKkwIiX1R0ZxESzpP9Xwp8-T0ou8fsLksygbKyH~3K1CyTHjTS51Ux-U-CjOPH9rtCOjjAaifdyMpK0PxW1fVdoGswFywTz9Q-6DUMsIu5TsPMF0-UO1Wn8vCpVAWbBJAOtKCfBrGzp-E~GCbfCNs5xY19nLobMD5ehjsBdI1lXwGDCQ7kBOwC58uuC3BOoazgrB6IrGskyMTexawtthO9mhuPm91bq4xhNaCYHAe059xg5emnM7jFBVzQgjaZ5lOLn~HqcWofJ7oc0doE6XI6kOo~YncBQAEAAcAAA==")
|
||||
// sNL
|
||||
new Destination("JC63wJNOqSJmymkj4~UJWywBTvDGikKMoYP0HX2Wz9c5l3otXSkwnxWAFL4cKr~Ygh3BNNi2t93vuLIiI1W8AsE42kR~PwRx~Y-WvIHXR6KUejRmOp-n8WidtjKg9k4aDy428uSOedqXDxys5mpoeQXwDsv1CoPTTwnmb1GWFy~oTGIsCguCl~aJWGnqiKarPO3GJQ~ev-NbvAQzUfC3HeP1e6pdI5CGGjExahTCID5UjpJw8GaDXWlGmYWWH303Xu4x-vAHQy1dJLsOBCn8dZravsn5BKJk~j0POUon45CCx-~NYtaPe0Itt9cMdD2ciC76Rep1D0X0sm1SjlSs8sZ52KmF3oaLZ6OzgI9QLMIyBUrfi41sK5I0qTuUVBAkvW1xr~L-20dYJ9TrbOaOb2-vDIfKaxVi6xQOuhgQDiSBhd3qv2m0xGu-BM9DQYfNA0FdMjnZmqjmji9RMavzQSsVFIbQGLbrLepiEFlb7TseCK5UtRp8TxnG7L4gbYevBQAEAAcAAA=="),
|
||||
// dark_trion
|
||||
new Destination("Gec9L29FVcQvYDgpcYuEYdltJn06PPoOWAcAM8Af-gDm~ehlrJcwlLXXs0hidq~yP2A0X7QcDi6i6shAfuEofTchxGJl8LRNqj9lio7WnB7cIixXWL~uCkD7Np5LMX0~akNX34oOb9RcBYVT2U5rFGJmJ7OtBv~IBkGeLhsMrqaCjahd0jdBO~QJ-t82ZKZhh044d24~JEfF9zSJxdBoCdAcXzryGNy7sYtFVDFsPKJudAxSW-UsSQiGw2~k-TxyF0r-iAt1IdzfNu8Lu0WPqLdhDYJWcPldx2PR5uJorI~zo~z3I5RX3NwzarlbD4nEP5s65ahPSfVCEkzmaJUBgP8DvBqlFaX89K4nGRYc7jkEjJ8cX4L6YPXUpTPWcfKkW259WdQY3YFh6x7rzijrGZewpczOLCrt-bZRYgDrUibmZxKZmNhy~lQu4gYVVjkz1i4tL~DWlhIc4y0x2vItwkYLArPPi~ejTnt-~Lhb7oPMXRcWa3UrwGKpFvGZY4NXBQAEAAcAAA==")
|
||||
]
|
||||
|
||||
static List<Destination> getCacheServers() {
|
||||
|
||||
@ -7,19 +7,17 @@ class Host {
|
||||
private static final int MAX_FAILURES = 3
|
||||
|
||||
final Destination destination
|
||||
private final int clearInterval, hopelessInterval, rejectionInterval, purgeInterval
|
||||
private final int clearInterval, hopelessInterval, rejectionInterval
|
||||
int failures,successes
|
||||
long lastAttempt
|
||||
long lastSuccessfulAttempt
|
||||
long lastRejection
|
||||
|
||||
public Host(Destination destination, int clearInterval, int hopelessInterval, int rejectionInterval,
|
||||
int purgeInterval) {
|
||||
public Host(Destination destination, int clearInterval, int hopelessInterval, int rejectionInterval) {
|
||||
this.destination = destination
|
||||
this.clearInterval = clearInterval
|
||||
this.hopelessInterval = hopelessInterval
|
||||
this.rejectionInterval = rejectionInterval
|
||||
this.purgeInterval = purgeInterval
|
||||
}
|
||||
|
||||
private void connectSuccessful() {
|
||||
@ -56,22 +54,17 @@ class Host {
|
||||
failures = 0
|
||||
}
|
||||
|
||||
synchronized boolean canTryAgain(final long now) {
|
||||
synchronized boolean canTryAgain() {
|
||||
lastSuccessfulAttempt > 0 &&
|
||||
now - lastAttempt > (clearInterval * 60 * 1000)
|
||||
System.currentTimeMillis() - lastAttempt > (clearInterval * 60 * 1000)
|
||||
}
|
||||
|
||||
synchronized boolean isHopeless(final long now) {
|
||||
synchronized boolean isHopeless() {
|
||||
isFailed() &&
|
||||
now - lastSuccessfulAttempt > (hopelessInterval * 60 * 1000)
|
||||
System.currentTimeMillis() - lastSuccessfulAttempt > (hopelessInterval * 60 * 1000)
|
||||
}
|
||||
|
||||
synchronized boolean isRecentlyRejected(final long now) {
|
||||
now - lastRejection < (rejectionInterval * 60 * 1000)
|
||||
}
|
||||
|
||||
synchronized boolean shouldBeForgotten(final long now) {
|
||||
isHopeless(now) &&
|
||||
now - lastAttempt > (purgeInterval * 60 * 1000)
|
||||
synchronized boolean isRecentlyRejected() {
|
||||
System.currentTimeMillis() - lastRejection < (rejectionInterval * 60 * 1000)
|
||||
}
|
||||
}
|
||||
|
||||
@ -52,8 +52,7 @@ class HostCache extends Service {
|
||||
hosts.get(e.destination).clearFailures()
|
||||
return
|
||||
}
|
||||
Host host = new Host(e.destination, settings.hostClearInterval, settings.hostHopelessInterval,
|
||||
settings.hostRejectInterval, settings.hostHopelessPurgeInterval)
|
||||
Host host = new Host(e.destination, settings.hostClearInterval, settings.hostHopelessInterval, settings.hostRejectInterval)
|
||||
if (allowHost(host)) {
|
||||
hosts.put(e.destination, host)
|
||||
}
|
||||
@ -65,8 +64,7 @@ class HostCache extends Service {
|
||||
Destination dest = e.endpoint.destination
|
||||
Host host = hosts.get(dest)
|
||||
if (host == null) {
|
||||
host = new Host(dest, settings.hostClearInterval, settings.hostHopelessInterval,
|
||||
settings.hostRejectInterval, settings.hostHopelessPurgeInterval)
|
||||
host = new Host(dest, settings.hostClearInterval, settings.hostHopelessInterval, settings.hostRejectInterval)
|
||||
hosts.put(dest, host)
|
||||
}
|
||||
|
||||
@ -86,10 +84,9 @@ class HostCache extends Service {
|
||||
List<Destination> getHosts(int n) {
|
||||
List<Destination> rv = new ArrayList<>(hosts.keySet())
|
||||
rv.retainAll {allowHost(hosts[it])}
|
||||
final long now = System.currentTimeMillis()
|
||||
rv.removeAll {
|
||||
def h = hosts[it];
|
||||
(h.isFailed() && !h.canTryAgain(now)) || h.isRecentlyRejected(now) || h.isHopeless(now)
|
||||
(h.isFailed() && !h.canTryAgain()) || h.isRecentlyRejected()
|
||||
}
|
||||
if (rv.size() <= n)
|
||||
return rv
|
||||
@ -119,9 +116,8 @@ class HostCache extends Service {
|
||||
|
||||
int countHopelessHosts() {
|
||||
List<Destination> rv = new ArrayList<>(hosts.keySet())
|
||||
final long now = System.currentTimeMillis()
|
||||
rv.retainAll {
|
||||
hosts[it].isHopeless(now)
|
||||
hosts[it].isHopeless()
|
||||
}
|
||||
rv.size()
|
||||
}
|
||||
@ -132,8 +128,7 @@ class HostCache extends Service {
|
||||
storage.eachLine {
|
||||
def entry = slurper.parseText(it)
|
||||
Destination dest = new Destination(entry.destination)
|
||||
Host host = new Host(dest, settings.hostClearInterval, settings.hostHopelessInterval,
|
||||
settings.hostRejectInterval, settings.hostHopelessPurgeInterval)
|
||||
Host host = new Host(dest, settings.hostClearInterval, settings.hostHopelessInterval, settings.hostRejectInterval)
|
||||
host.failures = Integer.valueOf(String.valueOf(entry.failures))
|
||||
host.successes = Integer.valueOf(String.valueOf(entry.successes))
|
||||
if (entry.lastAttempt != null)
|
||||
@ -166,12 +161,10 @@ class HostCache extends Service {
|
||||
}
|
||||
|
||||
private void save() {
|
||||
final long now = System.currentTimeMillis()
|
||||
hosts.keySet().removeAll { hosts[it].shouldBeForgotten(now) }
|
||||
storage.delete()
|
||||
storage.withPrintWriter { writer ->
|
||||
hosts.each { dest, host ->
|
||||
if (allowHost(host) && !host.isHopeless(now)) {
|
||||
if (allowHost(host) && !host.isHopeless()) {
|
||||
def map = [:]
|
||||
map.destination = dest.toBase64()
|
||||
map.failures = host.failures
|
||||
|
||||
@ -1,68 +1,28 @@
|
||||
package com.muwire.core.mesh
|
||||
|
||||
import java.util.concurrent.ConcurrentHashMap
|
||||
import java.util.stream.Collectors
|
||||
|
||||
import com.muwire.core.InfoHash
|
||||
import com.muwire.core.Persona
|
||||
import com.muwire.core.download.Pieces
|
||||
import com.muwire.core.util.DataUtil
|
||||
|
||||
import net.i2p.data.Base64
|
||||
import net.i2p.data.Destination
|
||||
import net.i2p.util.ConcurrentHashSet
|
||||
|
||||
/**
|
||||
* Representation of a download mesh.
|
||||
*
|
||||
* Two data structures - collection of known sources and collection of sources
|
||||
* we have successfully transferred data with.
|
||||
*
|
||||
* @author zab
|
||||
*/
|
||||
class Mesh {
|
||||
private final InfoHash infoHash
|
||||
private final Map<Destination,Persona> sources = new HashMap<>()
|
||||
private final Set<Destination> verified = new HashSet<>()
|
||||
final Pieces pieces
|
||||
private final Set<Persona> sources = new ConcurrentHashSet<>()
|
||||
private final Pieces pieces
|
||||
|
||||
Mesh(InfoHash infoHash, Pieces pieces) {
|
||||
this.infoHash = infoHash
|
||||
this.pieces = pieces
|
||||
}
|
||||
|
||||
synchronized Set<Persona> getRandom(int n, Persona exclude) {
|
||||
List<Destination> tmp = new ArrayList<>(verified)
|
||||
if (exclude != null)
|
||||
tmp.remove(exclude.destination)
|
||||
tmp.retainAll(sources.keySet()) // verified may contain nodes not in sources
|
||||
Set<Persona> getRandom(int n, Persona exclude) {
|
||||
List<Persona> tmp = new ArrayList<>(sources)
|
||||
tmp.remove(exclude)
|
||||
Collections.shuffle(tmp)
|
||||
if (tmp.size() > n)
|
||||
tmp = tmp[0..n-1]
|
||||
tmp.collect(new HashSet<>(), { sources[it] })
|
||||
}
|
||||
|
||||
synchronized void add(Persona persona) {
|
||||
sources.put(persona.destination, persona)
|
||||
}
|
||||
|
||||
synchronized void verify(Destination d) {
|
||||
verified.add(d)
|
||||
}
|
||||
|
||||
synchronized def toJson() {
|
||||
def json = [:]
|
||||
json.timestamp = System.currentTimeMillis()
|
||||
json.infoHash = Base64.encode(infoHash.getRoot())
|
||||
|
||||
Set<Persona> toPersist = new HashSet<>(sources.values())
|
||||
toPersist.retainAll { verified.contains(it.destination) }
|
||||
json.sources = toPersist.collect {it.toBase64()}
|
||||
json.nPieces = pieces.nPieces
|
||||
List<Integer> downloaded = pieces.getDownloaded()
|
||||
if( downloaded.size() > pieces.nPieces)
|
||||
return null
|
||||
json.xHave = DataUtil.encodeXHave(downloaded, pieces.nPieces)
|
||||
json
|
||||
if (tmp.size() < n)
|
||||
return tmp
|
||||
tmp[0..n-1]
|
||||
}
|
||||
}
|
||||
|
||||
@ -9,7 +9,6 @@ import com.muwire.core.MuWireSettings
|
||||
import com.muwire.core.Persona
|
||||
import com.muwire.core.download.Pieces
|
||||
import com.muwire.core.download.SourceDiscoveredEvent
|
||||
import com.muwire.core.download.SourceVerifiedEvent
|
||||
import com.muwire.core.files.FileManager
|
||||
import com.muwire.core.util.DataUtil
|
||||
|
||||
@ -57,25 +56,25 @@ class MeshManager {
|
||||
Mesh mesh = meshes.get(e.infoHash)
|
||||
if (mesh == null)
|
||||
return
|
||||
mesh.add(e.source)
|
||||
}
|
||||
|
||||
void onSourceVerifiedEvent(SourceVerifiedEvent e) {
|
||||
Mesh mesh = meshes.get(e.infoHash)
|
||||
if (mesh == null)
|
||||
return
|
||||
mesh.verify(e.source)
|
||||
mesh.sources.add(e.source)
|
||||
save()
|
||||
}
|
||||
|
||||
|
||||
private void save() {
|
||||
File meshFile = new File(home, "mesh.json")
|
||||
synchronized(meshes) {
|
||||
meshFile.withPrintWriter { writer ->
|
||||
meshes.values().each { mesh ->
|
||||
def json = mesh.toJson()
|
||||
if (json != null)
|
||||
writer.println(JsonOutput.toJson(json))
|
||||
def json = [:]
|
||||
json.timestamp = System.currentTimeMillis()
|
||||
json.infoHash = Base64.encode(mesh.infoHash.getRoot())
|
||||
json.sources = mesh.sources.stream().map({it.toBase64()}).collect(Collectors.toList())
|
||||
json.nPieces = mesh.pieces.nPieces
|
||||
List<Integer> downloaded = mesh.pieces.getDownloaded()
|
||||
if( downloaded.size() > mesh.pieces.nPieces)
|
||||
return
|
||||
json.xHave = DataUtil.encodeXHave(downloaded, mesh.pieces.nPieces)
|
||||
writer.println(JsonOutput.toJson(json))
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -100,8 +99,7 @@ class MeshManager {
|
||||
Mesh mesh = new Mesh(infoHash, pieces)
|
||||
json.sources.each { source ->
|
||||
Persona persona = new Persona(new ByteArrayInputStream(Base64.decode(source)))
|
||||
mesh.add(persona)
|
||||
mesh.verify(persona.destination) // assume if persisted it was verified
|
||||
mesh.sources.add(persona)
|
||||
}
|
||||
|
||||
if (json.xHave != null) {
|
||||
|
||||
@ -1,30 +0,0 @@
|
||||
package com.muwire.core.search
|
||||
|
||||
import com.muwire.core.download.SourceVerifiedEvent
|
||||
import com.muwire.core.util.FixedSizeFIFOSet
|
||||
|
||||
import net.i2p.data.Destination
|
||||
|
||||
/**
|
||||
* Caches destinations that have recently responded to with results.
|
||||
*/
|
||||
class ResponderCache {
|
||||
|
||||
private final FixedSizeFIFOSet<Destination> cache
|
||||
|
||||
ResponderCache(int capacity) {
|
||||
cache = new FixedSizeFIFOSet<>(capacity)
|
||||
}
|
||||
|
||||
synchronized void onUIResultBatchEvent(UIResultBatchEvent e) {
|
||||
cache.add(e.results[0].sender.destination)
|
||||
}
|
||||
|
||||
synchronized void onSourceVerifiedEvent(SourceVerifiedEvent e) {
|
||||
cache.add(e.source)
|
||||
}
|
||||
|
||||
synchronized boolean hasResponded(Destination d) {
|
||||
cache.contains(d)
|
||||
}
|
||||
}
|
||||
@ -9,11 +9,9 @@ import com.muwire.core.Persona
|
||||
import com.muwire.core.files.FileHasher
|
||||
import com.muwire.core.util.DataUtil
|
||||
|
||||
import groovy.util.logging.Log
|
||||
import net.i2p.data.Base64
|
||||
import net.i2p.data.Destination
|
||||
|
||||
@Log
|
||||
class ResultsParser {
|
||||
public static UIResultEvent parse(Persona p, UUID uuid, def json) throws InvalidSearchResultException {
|
||||
if (json.type != "Result")
|
||||
@ -105,8 +103,6 @@ class ResultsParser {
|
||||
int certificates = 0
|
||||
if (json.certificates != null)
|
||||
certificates = json.certificates
|
||||
|
||||
log.fine("Received result from ${p.getHumanReadableName()} name \"$name\" infoHash:\"${json.infohash}\"")
|
||||
|
||||
return new UIResultEvent( sender : p,
|
||||
name : name,
|
||||
|
||||
@ -77,19 +77,18 @@ class ResultsSender {
|
||||
if (it.getComment() != null) {
|
||||
comment = DataUtil.readi18nString(Base64.decode(it.getComment()))
|
||||
}
|
||||
int certificates = certificateManager.getByInfoHash(new InfoHash(it.getRoot())).size()
|
||||
int certificates = certificateManager.getByInfoHash(it.getInfoHash()).size()
|
||||
def uiResultEvent = new UIResultEvent( sender : me,
|
||||
name : it.getFile().getName(),
|
||||
size : length,
|
||||
infohash : new InfoHash(it.getRoot()),
|
||||
infohash : it.getInfoHash(),
|
||||
pieceSize : pieceSize,
|
||||
uuid : uuid,
|
||||
browse : settings.browseFiles,
|
||||
sources : suggested,
|
||||
comment : comment,
|
||||
certificates : certificates,
|
||||
chat : chatServer.isRunning() && settings.advertiseChat,
|
||||
feed : settings.fileFeed && settings.advertiseFeed
|
||||
chat : chatServer.running.get() && settings.advertiseChat
|
||||
)
|
||||
uiResultEvents << uiResultEvent
|
||||
}
|
||||
@ -120,7 +119,7 @@ class ResultsSender {
|
||||
me.write(os)
|
||||
os.writeShort((short)results.length)
|
||||
results.each {
|
||||
int certificates = certificateManager.getByInfoHash(new InfoHash(it.getRoot())).size()
|
||||
int certificates = certificateManager.getByInfoHash(it.getInfoHash()).size()
|
||||
def obj = sharedFileToObj(it, settings.browseFiles, certificates)
|
||||
def json = jsonOutput.toJson(obj)
|
||||
os.writeShort((short)json.length())
|
||||
@ -137,14 +136,12 @@ class ResultsSender {
|
||||
os.write("RESULTS $uuid\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
os.write("Sender: ${me.toBase64()}\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
os.write("Count: $results.length\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
boolean chat = chatServer.isRunning() && settings.advertiseChat
|
||||
boolean chat = chatServer.running.get() && settings.advertiseChat
|
||||
os.write("Chat: $chat\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
boolean feed = settings.fileFeed && settings.advertiseFeed
|
||||
os.write("Feed: $feed\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
os.write("\r\n".getBytes(StandardCharsets.US_ASCII))
|
||||
DataOutputStream dos = new DataOutputStream(new GZIPOutputStream(os))
|
||||
results.each {
|
||||
int certificates = certificateManager.getByInfoHash(new InfoHash(it.getRoot())).size()
|
||||
int certificates = certificateManager.getByInfoHash(it.getInfoHash()).size()
|
||||
def obj = sharedFileToObj(it, settings.browseFiles, certificates)
|
||||
def json = jsonOutput.toJson(obj)
|
||||
dos.writeShort((short)json.length())
|
||||
@ -173,7 +170,7 @@ class ResultsSender {
|
||||
obj.type = "Result"
|
||||
obj.version = 2
|
||||
obj.name = encodedName
|
||||
obj.infohash = Base64.encode(sf.getRoot())
|
||||
obj.infohash = Base64.encode(sf.getInfoHash().getRoot())
|
||||
obj.size = sf.getCachedLength()
|
||||
obj.pieceSize = sf.getPieceSize()
|
||||
|
||||
|
||||
@ -18,8 +18,7 @@ class UIResultEvent extends Event {
|
||||
boolean browse
|
||||
int certificates
|
||||
boolean chat
|
||||
boolean feed
|
||||
|
||||
|
||||
@Override
|
||||
public String toString() {
|
||||
super.toString() + "name:$name size:$size sender:${sender.getHumanReadableName()} pieceSize $pieceSize"
|
||||
|
||||
@ -1,218 +0,0 @@
|
||||
package com.muwire.core.tracker
|
||||
|
||||
import java.util.concurrent.ConcurrentHashMap
|
||||
import java.util.logging.Level
|
||||
import java.util.stream.Collectors
|
||||
|
||||
import com.muwire.core.Constants
|
||||
import com.muwire.core.InfoHash
|
||||
import com.muwire.core.MuWireSettings
|
||||
import com.muwire.core.Persona
|
||||
import com.muwire.core.download.DownloadManager
|
||||
import com.muwire.core.download.Pieces
|
||||
import com.muwire.core.files.FileManager
|
||||
import com.muwire.core.mesh.Mesh
|
||||
import com.muwire.core.mesh.MeshManager
|
||||
import com.muwire.core.trust.TrustLevel
|
||||
import com.muwire.core.trust.TrustService
|
||||
import com.muwire.core.util.DataUtil
|
||||
|
||||
import groovy.json.JsonOutput
|
||||
import groovy.json.JsonSlurper
|
||||
import groovy.util.logging.Log
|
||||
import net.i2p.client.I2PSession
|
||||
import net.i2p.client.I2PSessionMuxedListener
|
||||
import net.i2p.client.SendMessageOptions
|
||||
import net.i2p.client.datagram.I2PDatagramDissector
|
||||
import net.i2p.client.datagram.I2PDatagramMaker
|
||||
import net.i2p.data.Base64
|
||||
|
||||
@Log
|
||||
class TrackerResponder {
|
||||
private final I2PSession i2pSession
|
||||
private final MuWireSettings muSettings
|
||||
private final FileManager fileManager
|
||||
private final DownloadManager downloadManager
|
||||
private final MeshManager meshManager
|
||||
private final TrustService trustService
|
||||
private final Persona me
|
||||
|
||||
private final Map<UUID,Long> uuids = new HashMap<>()
|
||||
private final Timer expireTimer = new Timer("tracker-responder-timer", true)
|
||||
|
||||
private static final long UUID_LIFETIME = 10 * 60 * 1000
|
||||
|
||||
private volatile boolean shutdown
|
||||
|
||||
TrackerResponder(I2PSession i2pSession, MuWireSettings muSettings,
|
||||
FileManager fileManager, DownloadManager downloadManager,
|
||||
MeshManager meshManager, TrustService trustService,
|
||||
Persona me) {
|
||||
this.i2pSession = i2pSession
|
||||
this.muSettings = muSettings
|
||||
this.fileManager = fileManager
|
||||
this.downloadManager = downloadManager
|
||||
this.meshManager = meshManager
|
||||
this.trustService = trustService
|
||||
this.me = me
|
||||
}
|
||||
|
||||
void start() {
|
||||
i2pSession.addMuxedSessionListener(new Listener(), I2PSession.PROTO_DATAGRAM, Constants.TRACKER_PORT)
|
||||
expireTimer.schedule({expireUUIDs()} as TimerTask, UUID_LIFETIME, UUID_LIFETIME)
|
||||
}
|
||||
|
||||
void stop() {
|
||||
shutdown = true
|
||||
expireTimer.cancel()
|
||||
}
|
||||
|
||||
private void expireUUIDs() {
|
||||
final long now = System.currentTimeMillis()
|
||||
synchronized(uuids) {
|
||||
for (Iterator<UUID> iter = uuids.keySet().iterator(); iter.hasNext();) {
|
||||
UUID uuid = iter.next();
|
||||
Long time = uuids.get(uuid)
|
||||
if (now - time > UUID_LIFETIME)
|
||||
iter.remove()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private void respond(host, json) {
|
||||
log.info("responding to host $host with json $json")
|
||||
|
||||
def message = JsonOutput.toJson(json)
|
||||
def maker = new I2PDatagramMaker(i2pSession)
|
||||
message = maker.makeI2PDatagram(message.bytes)
|
||||
def options = new SendMessageOptions()
|
||||
options.setSendLeaseSet(false)
|
||||
i2pSession.sendMessage(host, message, 0, message.length, I2PSession.PROTO_DATAGRAM, Constants.TRACKER_PORT, Constants.TRACKER_PORT, options)
|
||||
}
|
||||
|
||||
class Listener implements I2PSessionMuxedListener {
|
||||
|
||||
@Override
|
||||
public void messageAvailable(I2PSession session, int msgId, long size) {
|
||||
}
|
||||
|
||||
@Override
|
||||
public void messageAvailable(I2PSession session, int msgId, long size, int proto, int fromport, int toport) {
|
||||
if (proto != I2PSession.PROTO_DATAGRAM) {
|
||||
log.warning "Received unexpected protocol $proto"
|
||||
return
|
||||
}
|
||||
|
||||
byte[] payload = session.receiveMessage(msgId)
|
||||
def dissector = new I2PDatagramDissector()
|
||||
try {
|
||||
dissector.loadI2PDatagram(payload)
|
||||
def sender = dissector.getSender()
|
||||
|
||||
log.info("got a tracker datagram from ${sender.toBase32()}")
|
||||
|
||||
// if not trusted, just drop it
|
||||
TrustLevel trustLevel = trustService.getLevel(sender)
|
||||
|
||||
if (trustLevel == TrustLevel.DISTRUSTED ||
|
||||
(trustLevel == TrustLevel.NEUTRAL && !muSettings.allowUntrusted)) {
|
||||
log.info("dropping, untrusted")
|
||||
return
|
||||
}
|
||||
|
||||
payload = dissector.getPayload()
|
||||
def slurper = new JsonSlurper()
|
||||
def json = slurper.parse(payload)
|
||||
|
||||
if (json.type != "TrackerPing") {
|
||||
log.warning("unknown type $json.type")
|
||||
return
|
||||
}
|
||||
|
||||
def response = [:]
|
||||
response.type = "TrackerPong"
|
||||
response.me = me.toBase64()
|
||||
|
||||
if (json.infoHash == null) {
|
||||
log.warning("infoHash missing")
|
||||
return
|
||||
}
|
||||
|
||||
if (json.uuid == null) {
|
||||
log.warning("uuid missing")
|
||||
return
|
||||
}
|
||||
|
||||
UUID uuid = UUID.fromString(json.uuid)
|
||||
synchronized(uuids) {
|
||||
if (uuids.containsKey(uuid)) {
|
||||
log.warning("duplicate uuid $uuid")
|
||||
return
|
||||
}
|
||||
uuids.put(uuid, System.currentTimeMillis())
|
||||
}
|
||||
response.uuid = json.uuid
|
||||
|
||||
if (!muSettings.allowTracking) {
|
||||
response.code = 403
|
||||
respond(sender, response)
|
||||
return
|
||||
}
|
||||
|
||||
if (json.version != 1) {
|
||||
log.warning("unknown version $json.version")
|
||||
response.code = 400
|
||||
response.message = "I only support version 1"
|
||||
respond(sender,response)
|
||||
return
|
||||
}
|
||||
|
||||
byte[] infoHashBytes = Base64.decode(json.infoHash)
|
||||
InfoHash infoHash = new InfoHash(infoHashBytes)
|
||||
|
||||
log.info("servicing request for infoHash ${json.infoHash} with uuid ${json.uuid}")
|
||||
|
||||
if (!(fileManager.isShared(infoHash) || downloadManager.isDownloading(infoHash))) {
|
||||
response.code = 404
|
||||
respond(sender, response)
|
||||
return
|
||||
}
|
||||
|
||||
Mesh mesh = meshManager.get(infoHash)
|
||||
|
||||
if (fileManager.isShared(infoHash))
|
||||
response.code = 200
|
||||
else if (mesh != null) {
|
||||
response.code = 206
|
||||
Pieces pieces = mesh.getPieces()
|
||||
response.xHave = DataUtil.encodeXHave(pieces, pieces.getnPieces())
|
||||
}
|
||||
|
||||
if (mesh != null)
|
||||
response.altlocs = mesh.getRandom(10, me).stream().map({it.toBase64()}).collect(Collectors.toList())
|
||||
|
||||
respond(sender,response)
|
||||
} catch (Exception e) {
|
||||
log.log(Level.WARNING, "invalid datagram", e)
|
||||
}
|
||||
}
|
||||
|
||||
@Override
|
||||
public void reportAbuse(I2PSession session, int severity) {
|
||||
}
|
||||
|
||||
@Override
|
||||
public void disconnected(I2PSession session) {
|
||||
if (!shutdown)
|
||||
log.severe("Tracker Responder session disconnected")
|
||||
}
|
||||
|
||||
@Override
|
||||
public void errorOccurred(I2PSession session, String message, Throwable error) {
|
||||
log.log(Level.SEVERE, message, error)
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
|
||||
}
|
||||
@ -2,7 +2,6 @@ package com.muwire.core.update
|
||||
|
||||
import java.util.logging.Level
|
||||
|
||||
import com.muwire.core.Constants
|
||||
import com.muwire.core.EventBus
|
||||
import com.muwire.core.InfoHash
|
||||
import com.muwire.core.MuWireSettings
|
||||
@ -49,7 +48,6 @@ class UpdateClient {
|
||||
private volatile boolean updateDownloading
|
||||
|
||||
private volatile String text
|
||||
private volatile boolean shutdown
|
||||
|
||||
UpdateClient(EventBus eventBus, I2PSession session, String myVersion, MuWireSettings settings,
|
||||
FileManager fileManager, Persona me, SigningPrivateKey spk) {
|
||||
@ -65,12 +63,11 @@ class UpdateClient {
|
||||
}
|
||||
|
||||
void start() {
|
||||
session.addMuxedSessionListener(new Listener(), I2PSession.PROTO_DATAGRAM, Constants.UPDATE_PORT)
|
||||
session.addMuxedSessionListener(new Listener(), I2PSession.PROTO_DATAGRAM, 2)
|
||||
timer.schedule({checkUpdate()} as TimerTask, 60000, 60 * 60 * 1000)
|
||||
}
|
||||
|
||||
void stop() {
|
||||
shutdown = true
|
||||
timer.cancel()
|
||||
}
|
||||
|
||||
@ -86,7 +83,7 @@ class UpdateClient {
|
||||
}
|
||||
|
||||
void onFileDownloadedEvent(FileDownloadedEvent e) {
|
||||
if (e.infoHash != updateInfoHash)
|
||||
if (e.downloadedFile.infoHash != updateInfoHash)
|
||||
return
|
||||
updateDownloading = false
|
||||
eventBus.publish(new UpdateDownloadedEvent(version : version, signer : signer, text : text))
|
||||
@ -111,7 +108,7 @@ class UpdateClient {
|
||||
ping = maker.makeI2PDatagram(ping.bytes)
|
||||
def options = new SendMessageOptions()
|
||||
options.setSendLeaseSet(true)
|
||||
session.sendMessage(UpdateServers.UPDATE_SERVER, ping, 0, ping.length, I2PSession.PROTO_DATAGRAM, Constants.UPDATE_PORT, 0, options)
|
||||
session.sendMessage(UpdateServers.UPDATE_SERVER, ping, 0, ping.length, I2PSession.PROTO_DATAGRAM, 2, 0, options)
|
||||
}
|
||||
|
||||
class Listener implements I2PSessionMuxedListener {
|
||||
@ -201,8 +198,7 @@ class UpdateClient {
|
||||
|
||||
@Override
|
||||
public void disconnected(I2PSession session) {
|
||||
if (!shutdown)
|
||||
log.severe("I2P session disconnected")
|
||||
log.severe("I2P session disconnected")
|
||||
}
|
||||
|
||||
@Override
|
||||
|
||||
@ -3,5 +3,4 @@ package com.muwire.core.upload
|
||||
class ContentRequest extends Request {
|
||||
Range range
|
||||
int have
|
||||
boolean browse, feed, chat
|
||||
}
|
||||
|
||||
@ -106,7 +106,7 @@ class ContentUploader extends Uploader {
|
||||
return done ? 100 : 0
|
||||
int position = mapped.position()
|
||||
int total = request.getRange().end - request.getRange().start
|
||||
(int)(position * 100.0d / total)
|
||||
(int)(position * 100.0 / total)
|
||||
}
|
||||
|
||||
@Override
|
||||
@ -137,24 +137,4 @@ class ContentUploader extends Uploader {
|
||||
request.infoHash == other.request.infoHash &&
|
||||
request.getDownloader() == other.request.getDownloader()
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean isBrowseEnabled() {
|
||||
request.browse
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean isFeedEnabled() {
|
||||
request.feed
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean isChatEnabled() {
|
||||
request.chat
|
||||
}
|
||||
|
||||
@Override
|
||||
public Persona getDownloaderPersona() {
|
||||
request.downloader
|
||||
}
|
||||
}
|
||||
|
||||
@ -4,7 +4,6 @@ import java.nio.ByteBuffer
|
||||
import java.nio.charset.StandardCharsets
|
||||
|
||||
import com.muwire.core.InfoHash
|
||||
import com.muwire.core.Persona
|
||||
import com.muwire.core.connection.Endpoint
|
||||
|
||||
import net.i2p.data.Base64
|
||||
@ -46,7 +45,7 @@ class HashListUploader extends Uploader {
|
||||
|
||||
@Override
|
||||
public synchronized int getProgress() {
|
||||
(int)(mapped.position() * 100.0d / mapped.capacity())
|
||||
(int)(mapped.position() * 100.0 / mapped.capacity())
|
||||
}
|
||||
|
||||
@Override
|
||||
@ -76,24 +75,4 @@ class HashListUploader extends Uploader {
|
||||
HashListUploader other = (HashListUploader)o
|
||||
infoHash == other.infoHash && request.downloader == other.request.downloader
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean isBrowseEnabled() {
|
||||
return false;
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean isFeedEnabled() {
|
||||
return false;
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean isChatEnabled() {
|
||||
return false;
|
||||
}
|
||||
|
||||
@Override
|
||||
public Persona getDownloaderPersona() {
|
||||
request.downloader
|
||||
}
|
||||
}
|
||||
|
||||
@ -22,7 +22,7 @@ class Request {
|
||||
|
||||
static Request parseContentRequest(InfoHash infoHash, InputStream is) throws IOException {
|
||||
|
||||
Map<String, String> headers = DataUtil.readAllHeaders(is)
|
||||
Map<String, String> headers = parseHeaders(is)
|
||||
|
||||
if (!headers.containsKey("Range"))
|
||||
throw new IOException("Range header not found")
|
||||
@ -55,18 +55,12 @@ class Request {
|
||||
def encoded = headers["X-Have"].trim()
|
||||
have = DataUtil.decodeXHave(encoded).size()
|
||||
}
|
||||
|
||||
boolean browse = headers.containsKey("Browse") && Boolean.parseBoolean(headers['Browse'])
|
||||
boolean feed = headers.containsKey("Feed") && Boolean.parseBoolean(headers['Feed'])
|
||||
boolean chat = headers.containsKey("Chat") && Boolean.parseBoolean(headers['Chat'])
|
||||
|
||||
new ContentRequest( infoHash : infoHash, range : new Range(start, end),
|
||||
headers : headers, downloader : downloader, have : have,
|
||||
browse : browse, feed : feed, chat : chat)
|
||||
headers : headers, downloader : downloader, have : have)
|
||||
}
|
||||
|
||||
static Request parseHashListRequest(InfoHash infoHash, InputStream is) throws IOException {
|
||||
Map<String,String> headers = DataUtil.readAllHeaders(is)
|
||||
Map<String,String> headers = parseHeaders(is)
|
||||
Persona downloader = null
|
||||
if (headers.containsKey("X-Persona")) {
|
||||
def encoded = headers["X-Persona"].trim()
|
||||
@ -75,4 +69,55 @@ class Request {
|
||||
}
|
||||
new HashListRequest(infoHash : infoHash, headers : headers, downloader : downloader)
|
||||
}
|
||||
|
||||
private static Map<String, String> parseHeaders(InputStream is) {
|
||||
Map<String,String> headers = new HashMap<>()
|
||||
byte [] tmp = new byte[Constants.MAX_HEADER_SIZE]
|
||||
while(headers.size() < Constants.MAX_HEADERS) {
|
||||
boolean r = false
|
||||
boolean n = false
|
||||
int idx = 0
|
||||
while (true) {
|
||||
byte read = is.read()
|
||||
if (read == -1)
|
||||
throw new IOException("Stream closed")
|
||||
|
||||
if (!r && read == N)
|
||||
throw new IOException("Received N before R")
|
||||
if (read == R) {
|
||||
if (r)
|
||||
throw new IOException("double R")
|
||||
r = true
|
||||
continue
|
||||
}
|
||||
|
||||
if (r && !n) {
|
||||
if (read != N)
|
||||
throw new IOException("R not followed by N")
|
||||
n = true
|
||||
break
|
||||
}
|
||||
if (idx == 0x1 << 14)
|
||||
throw new IOException("Header too long")
|
||||
tmp[idx++] = read
|
||||
}
|
||||
|
||||
if (idx == 0)
|
||||
break
|
||||
|
||||
String header = new String(tmp, 0, idx, StandardCharsets.US_ASCII)
|
||||
log.fine("Read header $header")
|
||||
|
||||
int keyIdx = header.indexOf(":")
|
||||
if (keyIdx < 1)
|
||||
throw new IOException("Header key not found")
|
||||
if (keyIdx == header.length())
|
||||
throw new IOException("Header value not found")
|
||||
String key = header.substring(0, keyIdx)
|
||||
String value = header.substring(keyIdx + 1)
|
||||
headers.put(key, value)
|
||||
}
|
||||
headers
|
||||
}
|
||||
|
||||
}
|
||||
|
||||
@ -11,9 +11,7 @@ import com.muwire.core.connection.Endpoint
|
||||
import com.muwire.core.download.DownloadManager
|
||||
import com.muwire.core.download.Downloader
|
||||
import com.muwire.core.download.SourceDiscoveredEvent
|
||||
import com.muwire.core.download.SourceVerifiedEvent
|
||||
import com.muwire.core.files.FileManager
|
||||
import com.muwire.core.files.PersisterFolderService
|
||||
import com.muwire.core.mesh.Mesh
|
||||
import com.muwire.core.mesh.MeshManager
|
||||
|
||||
@ -24,7 +22,6 @@ import net.i2p.data.Base64
|
||||
public class UploadManager {
|
||||
private final EventBus eventBus
|
||||
private final FileManager fileManager
|
||||
private final PersisterFolderService persisterService
|
||||
private final MeshManager meshManager
|
||||
private final DownloadManager downloadManager
|
||||
private final MuWireSettings props
|
||||
@ -37,11 +34,9 @@ public class UploadManager {
|
||||
|
||||
public UploadManager(EventBus eventBus, FileManager fileManager,
|
||||
MeshManager meshManager, DownloadManager downloadManager,
|
||||
PersisterFolderService persisterService,
|
||||
MuWireSettings props) {
|
||||
this.eventBus = eventBus
|
||||
this.fileManager = fileManager
|
||||
this.persisterService = persisterService
|
||||
this.meshManager = meshManager
|
||||
this.downloadManager = downloadManager
|
||||
this.props = props
|
||||
@ -124,7 +119,6 @@ public class UploadManager {
|
||||
eventBus.publish(new UploadEvent(uploader : uploader))
|
||||
try {
|
||||
uploader.respond()
|
||||
eventBus.publish(new SourceVerifiedEvent(infoHash : request.infoHash, source : request.downloader.destination))
|
||||
} finally {
|
||||
decrementUploads(request.downloader)
|
||||
eventBus.publish(new UploadFinishedEvent(uploader : uploader))
|
||||
@ -168,7 +162,7 @@ public class UploadManager {
|
||||
|
||||
InfoHash fullInfoHash
|
||||
if (downloader == null) {
|
||||
fullInfoHash = persisterService.loadInfoHash(sharedFiles.iterator().next())
|
||||
fullInfoHash = sharedFiles.iterator().next().infoHash
|
||||
} else {
|
||||
byte [] hashList = downloader.getInfoHash().getHashList()
|
||||
if (hashList != null && hashList.length > 0)
|
||||
@ -261,7 +255,6 @@ public class UploadManager {
|
||||
eventBus.publish(new UploadEvent(uploader : uploader))
|
||||
try {
|
||||
uploader.respond()
|
||||
eventBus.publish(new SourceVerifiedEvent(infoHash : request.infoHash, source : request.downloader.destination))
|
||||
} finally {
|
||||
eventBus.publish(new UploadFinishedEvent(uploader : uploader))
|
||||
}
|
||||
|
||||
@ -6,7 +6,6 @@ import java.nio.charset.StandardCharsets
|
||||
import java.nio.file.Files
|
||||
import java.nio.file.StandardOpenOption
|
||||
|
||||
import com.muwire.core.Persona
|
||||
import com.muwire.core.connection.Endpoint
|
||||
|
||||
abstract class Uploader {
|
||||
@ -32,7 +31,6 @@ abstract class Uploader {
|
||||
}
|
||||
|
||||
abstract String getName();
|
||||
|
||||
|
||||
/**
|
||||
* @return an integer between 0 and 100
|
||||
@ -40,7 +38,6 @@ abstract class Uploader {
|
||||
abstract int getProgress();
|
||||
|
||||
abstract String getDownloader();
|
||||
abstract Persona getDownloaderPersona();
|
||||
|
||||
abstract int getDonePieces();
|
||||
|
||||
@ -48,15 +45,11 @@ abstract class Uploader {
|
||||
|
||||
abstract long getTotalSize();
|
||||
|
||||
abstract boolean isBrowseEnabled();
|
||||
abstract boolean isFeedEnabled();
|
||||
abstract boolean isChatEnabled();
|
||||
|
||||
synchronized int speed() {
|
||||
final long now = System.currentTimeMillis()
|
||||
long interval = Math.max(1000, now - lastSpeedRead)
|
||||
lastSpeedRead = now;
|
||||
int currSpeed = (int) (dataSinceLastRead * 1000.0d / interval)
|
||||
int currSpeed = (int) (dataSinceLastRead * 1000.0 / interval)
|
||||
dataSinceLastRead = 0
|
||||
|
||||
// normalize to speedArr.size
|
||||
|
||||
@ -4,8 +4,6 @@ import net.i2p.crypto.SigType;
|
||||
|
||||
public class Constants {
|
||||
public static final byte PERSONA_VERSION = (byte)1;
|
||||
public static final String INVALID_NICKNAME_CHARS = "'\"();<>=@$%";
|
||||
public static final int MAX_NICKNAME_LENGTH = 30;
|
||||
public static final byte FILE_CERT_VERSION = (byte)2;
|
||||
public static final int CHAT_VERSION = 1;
|
||||
|
||||
@ -19,8 +17,5 @@ public class Constants {
|
||||
|
||||
public static final int MAX_COMMENT_LENGTH = 0x1 << 15;
|
||||
|
||||
public static final long MAX_QUERY_AGE = 5 * 60 * 1000L;
|
||||
|
||||
public static final int UPDATE_PORT = 2;
|
||||
public static final int TRACKER_PORT = 3;
|
||||
public static final long MAX_QUERY_AGE = 5 * 60 * 1000L;
|
||||
}
|
||||
|
||||
@ -10,9 +10,9 @@ public class DownloadedFile extends SharedFile {
|
||||
|
||||
private final Set<Destination> sources;
|
||||
|
||||
public DownloadedFile(File file, byte[] root, int pieceSize, Set<Destination> sources)
|
||||
public DownloadedFile(File file, InfoHash infoHash, int pieceSize, Set<Destination> sources)
|
||||
throws IOException {
|
||||
super(file, root, pieceSize);
|
||||
super(file, infoHash, pieceSize);
|
||||
this.sources = sources;
|
||||
}
|
||||
|
||||
|
||||
@ -1,25 +0,0 @@
|
||||
package com.muwire.core;
|
||||
|
||||
public class InvalidNicknameException extends Exception {
|
||||
|
||||
public InvalidNicknameException() {
|
||||
}
|
||||
|
||||
public InvalidNicknameException(String message) {
|
||||
super(message);
|
||||
}
|
||||
|
||||
public InvalidNicknameException(Throwable cause) {
|
||||
super(cause);
|
||||
}
|
||||
|
||||
public InvalidNicknameException(String message, Throwable cause) {
|
||||
super(message, cause);
|
||||
}
|
||||
|
||||
public InvalidNicknameException(String message, Throwable cause, boolean enableSuppression,
|
||||
boolean writableStackTrace) {
|
||||
super(message, cause, enableSuppression, writableStackTrace);
|
||||
}
|
||||
|
||||
}
|
||||
@ -7,8 +7,6 @@ import java.io.IOException;
|
||||
import java.io.InputStream;
|
||||
import java.io.OutputStream;
|
||||
|
||||
import com.muwire.core.util.DataUtil;
|
||||
|
||||
import net.i2p.crypto.DSAEngine;
|
||||
import net.i2p.data.Base64;
|
||||
import net.i2p.data.DataFormatException;
|
||||
@ -27,15 +25,12 @@ public class Persona {
|
||||
private volatile String base64;
|
||||
private volatile byte[] payload;
|
||||
|
||||
public Persona(InputStream personaStream) throws IOException, DataFormatException, InvalidSignatureException, InvalidNicknameException {
|
||||
public Persona(InputStream personaStream) throws IOException, DataFormatException, InvalidSignatureException {
|
||||
version = (byte) (personaStream.read() & 0xFF);
|
||||
if (version != Constants.PERSONA_VERSION)
|
||||
throw new IOException("Unknown version "+version);
|
||||
|
||||
name = new Name(personaStream);
|
||||
if (!DataUtil.isValidName(name.name))
|
||||
throw new InvalidNicknameException(name.name + " is not a valid nickname");
|
||||
|
||||
destination = Destination.create(personaStream);
|
||||
sig = new byte[SIG_LEN];
|
||||
DataInputStream dis = new DataInputStream(personaStream);
|
||||
@ -43,7 +38,7 @@ public class Persona {
|
||||
if (!verify(version, name, destination, sig))
|
||||
throw new InvalidSignatureException(getHumanReadableName() + " didn't verify");
|
||||
}
|
||||
|
||||
|
||||
private static boolean verify(byte version, Name name, Destination destination, byte [] sig)
|
||||
throws IOException, DataFormatException {
|
||||
ByteArrayOutputStream baos = new ByteArrayOutputStream();
|
||||
@ -52,7 +47,7 @@ public class Persona {
|
||||
destination.writeBytes(baos);
|
||||
byte[] payload = baos.toByteArray();
|
||||
SigningPublicKey spk = destination.getSigningPublicKey();
|
||||
Signature signature = new Signature(spk.getType(), sig);
|
||||
Signature signature = new Signature(Constants.SIG_TYPE, sig);
|
||||
return DSAEngine.getInstance().verifySignature(signature, payload, spk);
|
||||
}
|
||||
|
||||
|
||||
@ -2,10 +2,7 @@ package com.muwire.core;
|
||||
|
||||
import java.io.File;
|
||||
import java.io.IOException;
|
||||
import java.security.MessageDigest;
|
||||
import java.security.NoSuchAlgorithmException;
|
||||
import java.util.ArrayList;
|
||||
import java.util.Arrays;
|
||||
import java.util.Collections;
|
||||
import java.util.HashSet;
|
||||
import java.util.List;
|
||||
@ -19,49 +16,44 @@ import net.i2p.data.Base64;
|
||||
public class SharedFile {
|
||||
|
||||
private final File file;
|
||||
private final byte[] root;
|
||||
private final InfoHash infoHash;
|
||||
private final int pieceSize;
|
||||
|
||||
private final String cachedPath;
|
||||
private final long cachedLength;
|
||||
|
||||
private String b64PathHash;
|
||||
|
||||
private final String b64EncodedFileName;
|
||||
private final String b64EncodedHashRoot;
|
||||
private final List<String> b64EncodedHashList;
|
||||
|
||||
private volatile String comment;
|
||||
private final Set<String> downloaders = Collections.synchronizedSet(new HashSet<>());
|
||||
private final Set<SearchEntry> searches = Collections.synchronizedSet(new HashSet<>());
|
||||
private volatile boolean published;
|
||||
private volatile long publishedTimestamp;
|
||||
|
||||
public SharedFile(File file, byte[] root, int pieceSize) throws IOException {
|
||||
public SharedFile(File file, InfoHash infoHash, int pieceSize) throws IOException {
|
||||
this.file = file;
|
||||
this.root = root;
|
||||
this.infoHash = infoHash;
|
||||
this.pieceSize = pieceSize;
|
||||
this.cachedPath = file.getAbsolutePath();
|
||||
this.cachedLength = file.length();
|
||||
this.b64EncodedFileName = Base64.encode(DataUtil.encodei18nString(file.toString()));
|
||||
this.b64EncodedHashRoot = Base64.encode(infoHash.getRoot());
|
||||
|
||||
List<String> b64List = new ArrayList<String>();
|
||||
byte[] tmp = new byte[32];
|
||||
for (int i = 0; i < infoHash.getHashList().length / 32; i++) {
|
||||
System.arraycopy(infoHash.getHashList(), i * 32, tmp, 0, 32);
|
||||
b64List.add(Base64.encode(tmp));
|
||||
}
|
||||
this.b64EncodedHashList = b64List;
|
||||
}
|
||||
|
||||
public File getFile() {
|
||||
return file;
|
||||
}
|
||||
|
||||
public byte[] getPathHash() throws NoSuchAlgorithmException {
|
||||
MessageDigest digester = MessageDigest.getInstance("SHA-256");
|
||||
digester.update(file.getAbsolutePath().getBytes());
|
||||
return digester.digest();
|
||||
}
|
||||
|
||||
public String getB64PathHash() throws NoSuchAlgorithmException {
|
||||
if(b64PathHash == null){
|
||||
b64PathHash = Base64.encode(getPathHash());
|
||||
}
|
||||
return b64PathHash;
|
||||
}
|
||||
|
||||
public byte[] getRoot() {
|
||||
return root;
|
||||
public InfoHash getInfoHash() {
|
||||
return infoHash;
|
||||
}
|
||||
|
||||
public int getPieceSize() {
|
||||
@ -81,6 +73,14 @@ public class SharedFile {
|
||||
return b64EncodedFileName;
|
||||
}
|
||||
|
||||
public String getB64EncodedHashRoot() {
|
||||
return b64EncodedHashRoot;
|
||||
}
|
||||
|
||||
public List<String> getB64EncodedHashList() {
|
||||
return b64EncodedHashList;
|
||||
}
|
||||
|
||||
public String getCachedPath() {
|
||||
return cachedPath;
|
||||
}
|
||||
@ -116,28 +116,10 @@ public class SharedFile {
|
||||
public void addDownloader(String name) {
|
||||
downloaders.add(name);
|
||||
}
|
||||
|
||||
public void publish(long timestamp) {
|
||||
published = true;
|
||||
publishedTimestamp = timestamp;
|
||||
}
|
||||
|
||||
public void unpublish() {
|
||||
published = false;
|
||||
publishedTimestamp = 0;
|
||||
}
|
||||
|
||||
public boolean isPublished() {
|
||||
return published;
|
||||
}
|
||||
|
||||
public long getPublishedTimestamp() {
|
||||
return publishedTimestamp;
|
||||
}
|
||||
|
||||
@Override
|
||||
public int hashCode() {
|
||||
return file.hashCode() ^ Arrays.hashCode(root);
|
||||
return file.hashCode() ^ infoHash.hashCode();
|
||||
}
|
||||
|
||||
@Override
|
||||
@ -145,7 +127,7 @@ public class SharedFile {
|
||||
if (!(o instanceof SharedFile))
|
||||
return false;
|
||||
SharedFile other = (SharedFile)o;
|
||||
return file.equals(other.file) && Arrays.equals(root, other.root);
|
||||
return file.equals(other.file) && infoHash.equals(other.infoHash);
|
||||
}
|
||||
|
||||
public static class SearchEntry {
|
||||
@ -159,18 +141,6 @@ public class SharedFile {
|
||||
this.query = query;
|
||||
}
|
||||
|
||||
public Persona getSearcher() {
|
||||
return searcher;
|
||||
}
|
||||
|
||||
public long getTimestamp() {
|
||||
return timestamp;
|
||||
}
|
||||
|
||||
public String getQuery() {
|
||||
return query;
|
||||
}
|
||||
|
||||
public int hashCode() {
|
||||
return Objects.hash(searcher) ^ Objects.hash(timestamp) ^ query.hashCode();
|
||||
}
|
||||
|
||||
@ -1,81 +0,0 @@
|
||||
package com.muwire.core.filefeeds;
|
||||
|
||||
import com.muwire.core.Persona;
|
||||
|
||||
public class Feed {
|
||||
|
||||
private final Persona publisher;
|
||||
|
||||
private long updateInterval;
|
||||
private long lastUpdated;
|
||||
private volatile long lastUpdateAttempt;
|
||||
private int itemsToKeep;
|
||||
private boolean autoDownload;
|
||||
private boolean sequential;
|
||||
private FeedFetchStatus status;
|
||||
|
||||
public Feed(Persona publisher) {
|
||||
this.publisher = publisher;
|
||||
this.status = FeedFetchStatus.IDLE;
|
||||
}
|
||||
|
||||
public long getUpdateInterval() {
|
||||
return updateInterval;
|
||||
}
|
||||
|
||||
public void setUpdateInterval(long updateInterval) {
|
||||
this.updateInterval = updateInterval;
|
||||
}
|
||||
|
||||
public long getLastUpdated() {
|
||||
return lastUpdated;
|
||||
}
|
||||
|
||||
public void setLastUpdated(long lastUpdated) {
|
||||
this.lastUpdated = lastUpdated;
|
||||
}
|
||||
|
||||
public int getItemsToKeep() {
|
||||
return itemsToKeep;
|
||||
}
|
||||
|
||||
public void setItemsToKeep(int itemsToKeep) {
|
||||
this.itemsToKeep = itemsToKeep;
|
||||
}
|
||||
|
||||
public boolean isAutoDownload() {
|
||||
return autoDownload;
|
||||
}
|
||||
|
||||
public void setAutoDownload(boolean autoDownload) {
|
||||
this.autoDownload = autoDownload;
|
||||
}
|
||||
|
||||
public Persona getPublisher() {
|
||||
return publisher;
|
||||
}
|
||||
|
||||
public void setStatus(FeedFetchStatus status) {
|
||||
this.status = status;
|
||||
}
|
||||
|
||||
public FeedFetchStatus getStatus() {
|
||||
return status;
|
||||
}
|
||||
|
||||
public void setSequential(boolean sequential) {
|
||||
this.sequential = sequential;
|
||||
}
|
||||
|
||||
public boolean isSequential() {
|
||||
return sequential;
|
||||
}
|
||||
|
||||
public void setLastUpdateAttempt(long lastUpdateAttempt) {
|
||||
this.lastUpdateAttempt = lastUpdateAttempt;
|
||||
}
|
||||
|
||||
public long getLastUpdateAttempt() {
|
||||
return lastUpdateAttempt;
|
||||
}
|
||||
}
|
||||
@ -1,19 +0,0 @@
|
||||
package com.muwire.core.filefeeds;
|
||||
|
||||
public enum FeedFetchStatus {
|
||||
IDLE(false),
|
||||
CONNECTING(true),
|
||||
FETCHING(true),
|
||||
FINISHED(false),
|
||||
FAILED(false);
|
||||
|
||||
private final boolean active;
|
||||
|
||||
FeedFetchStatus(boolean active) {
|
||||
this.active = active;
|
||||
}
|
||||
|
||||
public boolean isActive() {
|
||||
return active;
|
||||
}
|
||||
}
|
||||
@ -1,79 +0,0 @@
|
||||
package com.muwire.core.filefeeds;
|
||||
|
||||
import java.util.Objects;
|
||||
|
||||
import com.muwire.core.InfoHash;
|
||||
import com.muwire.core.Persona;
|
||||
|
||||
public class FeedItem {
|
||||
|
||||
private final Persona publisher;
|
||||
private final long timestamp;
|
||||
private final String name;
|
||||
private final long size;
|
||||
private final int pieceSize;
|
||||
private final InfoHash infoHash;
|
||||
private final int certificates;
|
||||
private final String comment;
|
||||
|
||||
public FeedItem(Persona publisher, long timestamp, String name, long size, int pieceSize, InfoHash infoHash,
|
||||
int certificates, String comment) {
|
||||
super();
|
||||
this.publisher = publisher;
|
||||
this.timestamp = timestamp;
|
||||
this.name = name;
|
||||
this.size = size;
|
||||
this.pieceSize = pieceSize;
|
||||
this.infoHash = infoHash;
|
||||
this.certificates = certificates;
|
||||
this.comment = comment;
|
||||
}
|
||||
|
||||
public Persona getPublisher() {
|
||||
return publisher;
|
||||
}
|
||||
|
||||
public long getTimestamp() {
|
||||
return timestamp;
|
||||
}
|
||||
|
||||
public String getName() {
|
||||
return name;
|
||||
}
|
||||
|
||||
public long getSize() {
|
||||
return size;
|
||||
}
|
||||
|
||||
public int getPieceSize() {
|
||||
return pieceSize;
|
||||
}
|
||||
|
||||
public InfoHash getInfoHash() {
|
||||
return infoHash;
|
||||
}
|
||||
|
||||
public int getCertificates() {
|
||||
return certificates;
|
||||
}
|
||||
|
||||
public String getComment() {
|
||||
return comment;
|
||||
}
|
||||
|
||||
@Override
|
||||
public int hashCode() {
|
||||
return Objects.hash(publisher, timestamp, name, infoHash);
|
||||
}
|
||||
|
||||
@Override
|
||||
public boolean equals(Object o) {
|
||||
if (!(o instanceof FeedItem))
|
||||
return false;
|
||||
FeedItem other = (FeedItem)o;
|
||||
return Objects.equals(publisher, other.publisher) &&
|
||||
timestamp == other.timestamp &&
|
||||
Objects.equals(name, other.name) &&
|
||||
Objects.equals(infoHash, other.infoHash);
|
||||
}
|
||||
}
|
||||
@ -1,30 +0,0 @@
|
||||
package com.muwire.core.filefeeds;
|
||||
|
||||
public class InvalidFeedItemException extends Exception {
|
||||
|
||||
public InvalidFeedItemException() {
|
||||
super();
|
||||
}
|
||||
|
||||
public InvalidFeedItemException(String message, Throwable cause, boolean enableSuppression,
|
||||
boolean writableStackTrace) {
|
||||
super(message, cause, enableSuppression, writableStackTrace);
|
||||
// TODO Auto-generated constructor stub
|
||||
}
|
||||
|
||||
public InvalidFeedItemException(String message, Throwable cause) {
|
||||
super(message, cause);
|
||||
// TODO Auto-generated constructor stub
|
||||
}
|
||||
|
||||
public InvalidFeedItemException(String message) {
|
||||
super(message);
|
||||
// TODO Auto-generated constructor stub
|
||||
}
|
||||
|
||||
public InvalidFeedItemException(Throwable cause) {
|
||||
super(cause);
|
||||
// TODO Auto-generated constructor stub
|
||||
}
|
||||
|
||||
}
|
||||
@ -58,9 +58,9 @@ public class DataUtil {
|
||||
if (header.length != 3)
|
||||
throw new IllegalArgumentException("header length $header.length");
|
||||
|
||||
return ((header[0] & 0x7F) << 16) |
|
||||
((header[1] & 0xFF) << 8) |
|
||||
(header[2] & 0xFF);
|
||||
return (((int)(header[0] & 0x7F)) << 16) |
|
||||
(((int)(header[1] & 0xFF) << 8)) |
|
||||
((int)header[2] & 0xFF);
|
||||
}
|
||||
|
||||
public static String readi18nString(byte [] encoded) {
|
||||
@ -174,7 +174,7 @@ public class DataUtil {
|
||||
clean.setAccessible(true);
|
||||
clean.invoke(cleaner.invoke(cb));
|
||||
} else {
|
||||
Class<?> unsafeClass;
|
||||
Class unsafeClass;
|
||||
try {
|
||||
unsafeClass = Class.forName("sun.misc.Unsafe");
|
||||
} catch(Exception ex) {
|
||||
@ -216,13 +216,4 @@ public class DataUtil {
|
||||
Signature sig = DSAEngine.getInstance().sign(payload, spk);
|
||||
return sig.getData();
|
||||
}
|
||||
|
||||
public static boolean isValidName(String name) {
|
||||
if (name.length() > Constants.MAX_NICKNAME_LENGTH)
|
||||
return false;
|
||||
for (int i = 0; i < Constants.INVALID_NICKNAME_CHARS.length(); i++)
|
||||
if (name.indexOf(Constants.INVALID_NICKNAME_CHARS.charAt(i)) >= 0)
|
||||
return false;
|
||||
return true;
|
||||
}
|
||||
}
|
||||
|
||||
@ -1,35 +0,0 @@
|
||||
package com.muwire.core.util;
|
||||
|
||||
import java.util.ArrayDeque;
|
||||
import java.util.Deque;
|
||||
import java.util.HashSet;
|
||||
import java.util.Set;
|
||||
|
||||
public class FixedSizeFIFOSet<T> {
|
||||
|
||||
private final int capacity;
|
||||
private final Set<T> set = new HashSet<>();
|
||||
private final Deque<T> fifo = new ArrayDeque<>();
|
||||
|
||||
public FixedSizeFIFOSet(final int capacity) {
|
||||
this.capacity = capacity;
|
||||
}
|
||||
|
||||
public boolean contains(T element) {
|
||||
return set.contains(element);
|
||||
}
|
||||
|
||||
public void add(T element) {
|
||||
if (!set.contains(element)) {
|
||||
if (set.size() == capacity) {
|
||||
T toRemove = fifo.removeLast();
|
||||
set.remove(toRemove);
|
||||
}
|
||||
fifo.addFirst(element);
|
||||
set.add(element);
|
||||
} else {
|
||||
fifo.remove(element);
|
||||
fifo.addFirst(element);
|
||||
}
|
||||
}
|
||||
}
|
||||
@ -46,7 +46,7 @@ class DownloadSessionTest {
|
||||
eventBus = new EventBus()
|
||||
}
|
||||
|
||||
private void initSession(int size, def claimedPieces = [], boolean browse = false, boolean feed = false, boolean chat = false) {
|
||||
private void initSession(int size, def claimedPieces = []) {
|
||||
Random r = new Random()
|
||||
byte [] content = new byte[size]
|
||||
r.nextBytes(content)
|
||||
@ -78,8 +78,7 @@ class DownloadSessionTest {
|
||||
toUploader = new PipedOutputStream(fromDownloader)
|
||||
endpoint = new Endpoint(null, fromUploader, toUploader, null)
|
||||
|
||||
session = new DownloadSession(eventBus, "",pieces, infoHash, endpoint, target, pieceSize, size, available, new AtomicLong(),
|
||||
browse, feed, chat)
|
||||
session = new DownloadSession(eventBus, "",pieces, infoHash, endpoint, target, pieceSize, size, available, new AtomicLong())
|
||||
downloadThread = new Thread( { perform() } as Runnable)
|
||||
downloadThread.setDaemon(true)
|
||||
downloadThread.start()
|
||||
|
||||
@ -39,13 +39,13 @@ class FileManagerTest {
|
||||
@Test
|
||||
void testHash1Result() {
|
||||
File f = new File("a b.c")
|
||||
byte [] root = new byte[32]
|
||||
SharedFile sf = new SharedFile(f,root, 0)
|
||||
InfoHash ih = InfoHash.fromHashList(new byte[32])
|
||||
SharedFile sf = new SharedFile(f,ih, 0)
|
||||
FileHashedEvent fhe = new FileHashedEvent(sharedFile: sf)
|
||||
manager.onFileHashedEvent(fhe)
|
||||
|
||||
UUID uuid = UUID.randomUUID()
|
||||
SearchEvent se = new SearchEvent(searchHash: root, uuid: uuid)
|
||||
SearchEvent se = new SearchEvent(searchHash: ih.getRoot(), uuid: uuid)
|
||||
|
||||
manager.onSearchEvent(se)
|
||||
Thread.sleep(20)
|
||||
@ -58,14 +58,14 @@ class FileManagerTest {
|
||||
|
||||
@Test
|
||||
void testHash2Results() {
|
||||
byte [] root = new byte[32]
|
||||
SharedFile sf1 = new SharedFile(new File("a b.c"), root, 0)
|
||||
SharedFile sf2 = new SharedFile(new File("d e.f"), root, 0)
|
||||
InfoHash ih = InfoHash.fromHashList(new byte[32])
|
||||
SharedFile sf1 = new SharedFile(new File("a b.c"), ih, 0)
|
||||
SharedFile sf2 = new SharedFile(new File("d e.f"), ih, 0)
|
||||
manager.onFileLoadedEvent new FileLoadedEvent(loadedFile : sf1)
|
||||
manager.onFileLoadedEvent new FileLoadedEvent(loadedFile : sf2)
|
||||
|
||||
UUID uuid = UUID.randomUUID()
|
||||
SearchEvent se = new SearchEvent(searchHash: root, uuid: uuid)
|
||||
SearchEvent se = new SearchEvent(searchHash: ih.getRoot(), uuid: uuid)
|
||||
|
||||
manager.onSearchEvent(se)
|
||||
Thread.sleep(20)
|
||||
@ -81,7 +81,7 @@ class FileManagerTest {
|
||||
void testHash0Results() {
|
||||
File f = new File("a b.c")
|
||||
InfoHash ih = InfoHash.fromHashList(new byte[32])
|
||||
SharedFile sf = new SharedFile(f,ih.getRoot(), 0)
|
||||
SharedFile sf = new SharedFile(f,ih, 0)
|
||||
FileHashedEvent fhe = new FileHashedEvent(sharedFile: sf)
|
||||
manager.onFileHashedEvent(fhe)
|
||||
|
||||
@ -95,7 +95,7 @@ class FileManagerTest {
|
||||
void testKeyword1Result() {
|
||||
File f = new File("a b.c")
|
||||
InfoHash ih = InfoHash.fromHashList(new byte[32])
|
||||
SharedFile sf = new SharedFile(f,ih.getRoot(),0)
|
||||
SharedFile sf = new SharedFile(f,ih,0)
|
||||
FileHashedEvent fhe = new FileHashedEvent(sharedFile: sf)
|
||||
manager.onFileHashedEvent(fhe)
|
||||
|
||||
@ -113,12 +113,12 @@ class FileManagerTest {
|
||||
void testKeyword2Results() {
|
||||
File f1 = new File("a b.c")
|
||||
InfoHash ih1 = InfoHash.fromHashList(new byte[32])
|
||||
SharedFile sf1 = new SharedFile(f1, ih1.getRoot(), 0)
|
||||
SharedFile sf1 = new SharedFile(f1, ih1, 0)
|
||||
manager.onFileLoadedEvent new FileLoadedEvent(loadedFile: sf1)
|
||||
|
||||
File f2 = new File("c d.e")
|
||||
InfoHash ih2 = InfoHash.fromHashList(new byte[64])
|
||||
SharedFile sf2 = new SharedFile(f2, ih2.getRoot(), 0)
|
||||
SharedFile sf2 = new SharedFile(f2, ih2, 0)
|
||||
manager.onFileLoadedEvent new FileLoadedEvent(loadedFile: sf2)
|
||||
|
||||
UUID uuid = UUID.randomUUID()
|
||||
@ -136,7 +136,7 @@ class FileManagerTest {
|
||||
void testKeyword0Results() {
|
||||
File f = new File("a b.c")
|
||||
InfoHash ih = InfoHash.fromHashList(new byte[32])
|
||||
SharedFile sf = new SharedFile(f,ih.getRoot(),0)
|
||||
SharedFile sf = new SharedFile(f,ih,0)
|
||||
FileHashedEvent fhe = new FileHashedEvent(sharedFile: sf)
|
||||
manager.onFileHashedEvent(fhe)
|
||||
|
||||
@ -149,8 +149,8 @@ class FileManagerTest {
|
||||
@Test
|
||||
void testRemoveFileExistingHash() {
|
||||
InfoHash ih = InfoHash.fromHashList(new byte[32])
|
||||
SharedFile sf1 = new SharedFile(new File("a b.c"), ih.getRoot(), 0)
|
||||
SharedFile sf2 = new SharedFile(new File("d e.f"), ih.getRoot(), 0)
|
||||
SharedFile sf1 = new SharedFile(new File("a b.c"), ih, 0)
|
||||
SharedFile sf2 = new SharedFile(new File("d e.f"), ih, 0)
|
||||
manager.onFileLoadedEvent new FileLoadedEvent(loadedFile : sf1)
|
||||
manager.onFileLoadedEvent new FileLoadedEvent(loadedFile : sf2)
|
||||
|
||||
@ -167,12 +167,12 @@ class FileManagerTest {
|
||||
void testRemoveFile() {
|
||||
File f1 = new File("a b.c")
|
||||
InfoHash ih1 = InfoHash.fromHashList(new byte[32])
|
||||
SharedFile sf1 = new SharedFile(f1, ih1.getRoot(), 0)
|
||||
SharedFile sf1 = new SharedFile(f1, ih1, 0)
|
||||
manager.onFileLoadedEvent new FileLoadedEvent(loadedFile: sf1)
|
||||
|
||||
File f2 = new File("c d.e")
|
||||
InfoHash ih2 = InfoHash.fromHashList(new byte[64])
|
||||
SharedFile sf2 = new SharedFile(f2, ih2.getRoot(), 0)
|
||||
SharedFile sf2 = new SharedFile(f2, ih2, 0)
|
||||
manager.onFileLoadedEvent new FileLoadedEvent(loadedFile: sf2)
|
||||
|
||||
manager.onFileUnsharedEvent new FileUnsharedEvent(deleted : true, unsharedFile: sf2)
|
||||
@ -198,7 +198,7 @@ class FileManagerTest {
|
||||
comment = Base64.encode(DataUtil.encodei18nString(comment))
|
||||
File f1 = new File("MuWire-0.5.10.AppImage")
|
||||
InfoHash ih1 = InfoHash.fromHashList(new byte[32])
|
||||
SharedFile sf1 = new SharedFile(f1, ih1.getRoot(), 0)
|
||||
SharedFile sf1 = new SharedFile(f1, ih1, 0)
|
||||
sf1.setComment(comment)
|
||||
|
||||
manager.onFileLoadedEvent(new FileLoadedEvent(loadedFile : sf1))
|
||||
@ -206,7 +206,7 @@ class FileManagerTest {
|
||||
|
||||
File f2 = new File("MuWire-0.6.0.AppImage")
|
||||
InfoHash ih2 = InfoHash.fromHashList(new byte[64])
|
||||
SharedFile sf2 = new SharedFile(f2, ih2.getRoot(), 0)
|
||||
SharedFile sf2 = new SharedFile(f2, ih2, 0)
|
||||
sf2.setComment(comment)
|
||||
|
||||
manager.onFileLoadedEvent(new FileLoadedEvent(loadedFile : sf2))
|
||||
|
||||
@ -45,7 +45,7 @@ class HasherServiceTest {
|
||||
def hashed = listener.poll()
|
||||
assert hashed instanceof FileHashedEvent
|
||||
assert hashed.sharedFile.file == f.getCanonicalFile()
|
||||
assert hashed.sharedFile.root != null
|
||||
assert hashed.sharedFile.infoHash != null
|
||||
assert listener.isEmpty()
|
||||
}
|
||||
|
||||
|
||||
@ -85,7 +85,7 @@ class PersisterServiceLoadingTest {
|
||||
def loadedFile = listener.publishedFiles[0]
|
||||
assert loadedFile != null
|
||||
assert loadedFile.file == sharedFile1.getCanonicalFile()
|
||||
assert loadedFile.root == ih1.getRoot()
|
||||
assert loadedFile.infoHash == ih1
|
||||
}
|
||||
|
||||
private static String getSharedFileJsonName(File sharedFile) {
|
||||
@ -128,7 +128,7 @@ class PersisterServiceLoadingTest {
|
||||
def loadedFile = listener.publishedFiles[0]
|
||||
assert loadedFile != null
|
||||
assert loadedFile.file == sharedFile1.getCanonicalFile()
|
||||
assert loadedFile.root == ih1.getRoot()
|
||||
assert loadedFile.infoHash == ih1
|
||||
}
|
||||
|
||||
@Test
|
||||
@ -169,10 +169,10 @@ class PersisterServiceLoadingTest {
|
||||
assert listener.publishedFiles.size() == 2
|
||||
def loadedFile1 = listener.publishedFiles[0]
|
||||
assert loadedFile1.file == sharedFile1.getCanonicalFile()
|
||||
assert loadedFile1.root == ih1.getRoot()
|
||||
assert loadedFile1.infoHash == ih1
|
||||
def loadedFile2 = listener.publishedFiles[1]
|
||||
assert loadedFile2.file == sharedFile2.getCanonicalFile()
|
||||
assert loadedFile2.root == ih2.getRoot()
|
||||
assert loadedFile2.infoHash == ih2
|
||||
}
|
||||
|
||||
@Test
|
||||
|
||||
@ -2,7 +2,6 @@ package com.muwire.core.files
|
||||
|
||||
import org.junit.After
|
||||
import org.junit.Before
|
||||
import org.junit.Ignore
|
||||
import org.junit.Test
|
||||
|
||||
import com.muwire.core.Destinations
|
||||
@ -17,7 +16,6 @@ import groovy.json.JsonSlurper
|
||||
import net.i2p.data.Base32
|
||||
import net.i2p.data.Base64
|
||||
|
||||
@Ignore
|
||||
class PersisterServiceSavingTest {
|
||||
|
||||
File f
|
||||
|
||||
@ -75,7 +75,6 @@ class HostCacheTest {
|
||||
settingsMock.ignore.getHostClearInterval { 0 }
|
||||
settingsMock.ignore.getHostHopelessInterval { 0 }
|
||||
settingsMock.ignore.getHostRejectInterval { 0 }
|
||||
settingsMock.ignore.getHostHopelessPurgeInterval { 0 }
|
||||
|
||||
initMocks()
|
||||
|
||||
@ -98,7 +97,6 @@ class HostCacheTest {
|
||||
settingsMock.ignore.getHostClearInterval { 0 }
|
||||
settingsMock.ignore.getHostHopelessInterval { 0 }
|
||||
settingsMock.ignore.getHostRejectInterval { 0 }
|
||||
settingsMock.ignore.getHostHopelessPurgeInterval { 0 }
|
||||
|
||||
initMocks()
|
||||
|
||||
@ -116,7 +114,6 @@ class HostCacheTest {
|
||||
settingsMock.ignore.getHostClearInterval { 0 }
|
||||
settingsMock.ignore.getHostHopelessInterval { 0 }
|
||||
settingsMock.ignore.getHostRejectInterval { 0 }
|
||||
settingsMock.ignore.getHostHopelessPurgeInterval { 0 }
|
||||
|
||||
initMocks()
|
||||
|
||||
@ -139,7 +136,6 @@ class HostCacheTest {
|
||||
settingsMock.ignore.getHostClearInterval { 0 }
|
||||
settingsMock.ignore.getHostHopelessInterval { 0 }
|
||||
settingsMock.ignore.getHostRejectInterval { 0 }
|
||||
settingsMock.ignore.getHostHopelessPurgeInterval { 0 }
|
||||
|
||||
initMocks()
|
||||
cache.onHostDiscoveredEvent(new HostDiscoveredEvent(destination: destinations.dest1))
|
||||
@ -164,7 +160,6 @@ class HostCacheTest {
|
||||
settingsMock.ignore.getHostClearInterval { 100 }
|
||||
settingsMock.ignore.getHostHopelessInterval { 0 }
|
||||
settingsMock.ignore.getHostRejectInterval { 0 }
|
||||
settingsMock.ignore.getHostHopelessPurgeInterval { 0 }
|
||||
|
||||
initMocks()
|
||||
cache.onHostDiscoveredEvent(new HostDiscoveredEvent(destination: destinations.dest1))
|
||||
@ -187,7 +182,6 @@ class HostCacheTest {
|
||||
settingsMock.ignore.getHostClearInterval { 0 }
|
||||
settingsMock.ignore.getHostHopelessInterval { 0 }
|
||||
settingsMock.ignore.getHostRejectInterval { 0 }
|
||||
settingsMock.ignore.getHostHopelessPurgeInterval { 0 }
|
||||
|
||||
initMocks()
|
||||
cache.onHostDiscoveredEvent(new HostDiscoveredEvent(destination: destinations.dest1))
|
||||
@ -217,7 +211,6 @@ class HostCacheTest {
|
||||
settingsMock.ignore.getHostClearInterval { 0 }
|
||||
settingsMock.ignore.getHostHopelessInterval { 0 }
|
||||
settingsMock.ignore.getHostRejectInterval { 0 }
|
||||
settingsMock.ignore.getHostHopelessPurgeInterval { 0 }
|
||||
|
||||
initMocks()
|
||||
cache.onHostDiscoveredEvent(new HostDiscoveredEvent(destination: destinations.dest1))
|
||||
@ -253,7 +246,6 @@ class HostCacheTest {
|
||||
settingsMock.ignore.getHostClearInterval { 0 }
|
||||
settingsMock.ignore.getHostHopelessInterval { 0 }
|
||||
settingsMock.ignore.getHostRejectInterval { 0 }
|
||||
settingsMock.ignore.getHostHopelessPurgeInterval { 0 }
|
||||
|
||||
initMocks()
|
||||
cache.onHostDiscoveredEvent(new HostDiscoveredEvent(destination: destinations.dest1))
|
||||
@ -274,7 +266,6 @@ class HostCacheTest {
|
||||
settingsMock.ignore.getHostClearInterval { 0 }
|
||||
settingsMock.ignore.getHostHopelessInterval { 0 }
|
||||
settingsMock.ignore.getHostRejectInterval { 0 }
|
||||
settingsMock.ignore.getHostHopelessPurgeInterval { 0 }
|
||||
|
||||
initMocks()
|
||||
cache.onHostDiscoveredEvent(new HostDiscoveredEvent(destination: destinations.dest1))
|
||||
@ -310,7 +301,6 @@ class HostCacheTest {
|
||||
settingsMock.ignore.getHostClearInterval { 0 }
|
||||
settingsMock.ignore.getHostHopelessInterval { 0 }
|
||||
settingsMock.ignore.getHostRejectInterval { 0 }
|
||||
settingsMock.ignore.getHostHopelessPurgeInterval { 0 }
|
||||
|
||||
initMocks()
|
||||
def rv = cache.getHosts(5)
|
||||
|
||||
@ -1,49 +0,0 @@
|
||||
package com.muwire.core.util
|
||||
|
||||
import org.junit.Test
|
||||
|
||||
class FixedSizeFIFOSetTest {
|
||||
|
||||
@Test
|
||||
public void testFifo() {
|
||||
FixedSizeFIFOSet<String> fifoSet = new FixedSizeFIFOSet(3);
|
||||
fifoSet.add("a")
|
||||
assert fifoSet.contains("a")
|
||||
|
||||
fifoSet.add("b")
|
||||
assert fifoSet.contains("a")
|
||||
assert fifoSet.contains("b")
|
||||
|
||||
fifoSet.add("c")
|
||||
assert fifoSet.contains("a")
|
||||
assert fifoSet.contains("b")
|
||||
assert fifoSet.contains("c")
|
||||
|
||||
fifoSet.add("d")
|
||||
assert !fifoSet.contains("a")
|
||||
assert fifoSet.contains("b")
|
||||
assert fifoSet.contains("c")
|
||||
assert fifoSet.contains("d")
|
||||
}
|
||||
|
||||
|
||||
@Test
|
||||
public void testDuplicateElement() {
|
||||
FixedSizeFIFOSet<String> fifoSet = new FixedSizeFIFOSet(3);
|
||||
|
||||
fifoSet.add("a")
|
||||
fifoSet.add("b")
|
||||
fifoSet.add("c")
|
||||
fifoSet.add("a")
|
||||
|
||||
assert fifoSet.contains("a")
|
||||
assert fifoSet.contains("b")
|
||||
assert fifoSet.contains("c")
|
||||
|
||||
fifoSet.add("d")
|
||||
assert fifoSet.contains("a")
|
||||
assert !fifoSet.contains("b")
|
||||
assert fifoSet.contains("c")
|
||||
assert fifoSet.contains("d")
|
||||
}
|
||||
}
|
||||
@ -1,49 +0,0 @@
|
||||
# MuWire Collections
|
||||
Status: Draft, Proposal, Unimplemented
|
||||
|
||||
MuWire collections are files containing meta-information about a grouping of files. They serve a similar purpose like .torrent files, but the internal format is rather different to account for the MuWire identity management.
|
||||
|
||||
A user wishing to create a collection of files needs to have shared all the files that are going to be part of the collection. Their full MuWire ID will be stored in the collection, so anyone wishing to download any of the files in the collection will try to download from them first.
|
||||
|
||||
The collection will be signed, so anyone can verify that the embedded full MuWire ID authored the collection.
|
||||
|
||||
### File Format
|
||||
|
||||
Header:
|
||||
|
||||
```
|
||||
byte 0: Collection version, currently fixed at "1".
|
||||
bytes 1,2 : unsigned 16-bit value of the number of files in the collection. Empty files or directories are not allowed.
|
||||
bytes 3-N: Full MuWire ID of the publisher of the collection, in Persona format.
|
||||
bytes N+1 to N+9: Timestamp of the collection, in milliseconds since epoch UTC
|
||||
bytes N+9 to M: Free-form description of the collection (comment). Format is UTF-8, maximum size is 32kb.
|
||||
```
|
||||
|
||||
The header is followed by a file entry for each file in the collection. The format is the follows:
|
||||
```
|
||||
byte 0: File entry version, currently fixed at "1".
|
||||
byte 1-33: hash of the file
|
||||
byte 34: Unsigned 8-bit number of path elements from root to where the file will ultimately be placed upon download.
|
||||
bytes 35-N : UTF-8 encoded length-prefixed path elements. Each element can be at most 32kb long. The last element is the name of the file.
|
||||
bytes N-M: free from description of the file (comment). Format is UTF-8, maximum size is 32kb.
|
||||
```
|
||||
|
||||
After the file entries follows a footer, which is simply a signature of the byte payload of the header and the file entries.
|
||||
|
||||
### Downloading
|
||||
|
||||
Since the collection is created from individual shared files, every file within the collection is searchable. It is possible to extend the shared file data structure to contain refererences to any collections the file belongs to - TBD.
|
||||
|
||||
When a user searches for a keyword or hash, they can find either the collection metafile itself or a file which is a member of one or more collections. In the latter case, the user is given the option to download the collection metafile.
|
||||
|
||||
If the user chooses to download the collection metafile, they will be presented with a dialog containing the metainformation contained in the collection descriptor. They will be able to see directory structure contained in the collection and will be able to choose individual files to download.
|
||||
|
||||
TBD - what happens when some of the files are already downloaded but are not in the final directory location?
|
||||
|
||||
Finally, when starting the download, the downloader always queries the persona in the collection first, regardless of who returned the search result.
|
||||
|
||||
### Sharing
|
||||
|
||||
When downloading the collection descriptor, the user makes the descriptor available for indexing. This way collection descriptors can propagate on the network.
|
||||
TBD - do they also index the comments and file names in the descriptor, even if they haven't downloaded the files?
|
||||
|
||||
@ -1,17 +1,15 @@
|
||||
group = com.muwire
|
||||
version = 0.7.4
|
||||
i2pVersion = 0.9.47
|
||||
groovyVersion = 3.0.4
|
||||
version = 0.6.8
|
||||
i2pVersion = 0.9.44
|
||||
groovyVersion = 2.4.15
|
||||
slf4jVersion = 1.7.25
|
||||
spockVersion = 1.1-groovy-2.4
|
||||
grailsVersion=4.0.0
|
||||
gorm.version=7.0.2.RELEASE
|
||||
griffonEnv=prod
|
||||
|
||||
# javac properties
|
||||
sourceCompatibility=1.8
|
||||
targetCompatibility=1.8
|
||||
compilerArgs=-Xlint:unchecked,cast,path,divzero,empty,path,finally,overrides
|
||||
|
||||
# plugin properties
|
||||
author = zab@mail.i2p
|
||||
@ -20,4 +18,4 @@ keystorePassword=changeit
|
||||
websiteURL=http://muwire.i2p
|
||||
updateURLsu3=http://muwire.i2p/MuWire-update.su3
|
||||
|
||||
pack200=false
|
||||
pack200=true
|
||||
|
||||
@ -42,25 +42,10 @@ griffon {
|
||||
|
||||
application {
|
||||
mainClassName = 'com.muwire.gui.Launcher'
|
||||
applicationDefaultJvmArgs = ['-Djava.util.logging.config.file=logging.properties','-Xmx256M']
|
||||
applicationName = 'MuWire'
|
||||
}
|
||||
|
||||
run {
|
||||
applicationDefaultJvmArgs=[]
|
||||
}
|
||||
|
||||
startScripts.doFirst {
|
||||
application.applicationDefaultJvmArgs = ["-Djava.util.logging.config.file=logging.properties",
|
||||
"-Xmx256M",
|
||||
"--add-opens", "java.base/java.lang=ALL-UNNAMED",
|
||||
"--add-opens", "java.base/sun.nio.fs=ALL-UNNAMED",
|
||||
"--add-opens", "java.base/java.nio=ALL-UNNAMED",
|
||||
"--add-opens", "java.desktop/java.awt=ALL-UNNAMED",
|
||||
"--add-opens", "java.desktop/javax.swing=ALL-UNNAMED",
|
||||
"--add-opens", "java.desktop/javax.swing.plaf.basic=ALL-UNNAMED"]
|
||||
|
||||
}
|
||||
|
||||
apply from: 'gradle/publishing.gradle'
|
||||
// apply from: 'gradle/code-coverage.gradle'
|
||||
// apply from: 'gradle/code-quality.gradle'
|
||||
@ -72,27 +57,9 @@ apply plugin: 'org.kordamp.gradle.stats'
|
||||
apply plugin: 'com.github.ben-manes.versions'
|
||||
apply plugin: 'com.github.kt3k.coveralls'
|
||||
|
||||
|
||||
configurations.all {
|
||||
exclude group:'org.codehaus.groovy', module:'groovy-test'
|
||||
exclude group:'org.codehaus.groovy', module:'groovy-testng'
|
||||
exclude group:'org.codehaus.groovy', module:'groovy-test-junit5'
|
||||
exclude group:'org.codehaus.groovy', module:'groovy-ant'
|
||||
exclude group:'org.codehaus.groovy', module:'groovy-sql'
|
||||
exclude group:'org.codehaus.groovy', module:'groovy-nio'
|
||||
exclude group:'org.codehaus.groovy', module:'groovy-servlet'
|
||||
exclude group:'org.codehaus.groovy', module:'groovy-jmx'
|
||||
exclude group:'org.codehaus.groovy', module:'groovy-groovydoc'
|
||||
exclude group:'org.codehaus.groovy', module:'groovy-groovysh'
|
||||
exclude group:'org.codehaus.groovy', module:'groovy-xml'
|
||||
exclude group:'org.codehaus.groovy', module:'groovy-docgenerator'
|
||||
// TODO: add more as discovered
|
||||
}
|
||||
|
||||
dependencies {
|
||||
compile project(":core")
|
||||
compile "org.codehaus.griffon:griffon-guice:${griffon.version}"
|
||||
compile "org.codehaus.groovy:groovy-all:${groovyVersion}"
|
||||
|
||||
// runtime "org.slf4j:slf4j-simple:${slf4jVersion}"
|
||||
|
||||
@ -101,9 +68,6 @@ dependencies {
|
||||
runtime group: 'org.slf4j', name: 'jul-to-slf4j', version: "${slf4jVersion}"
|
||||
runtime "javax.annotation:javax.annotation-api:1.3.2"
|
||||
|
||||
// because java 14 doesn't come with it
|
||||
runtime 'mrj:MRJToolkitStubs:1.0'
|
||||
|
||||
testCompile "org.codehaus.griffon:griffon-fest-test:${griffon.version}"
|
||||
testCompile "org.spockframework:spock-core:${spockVersion}"
|
||||
testCompile('org.awaitility:awaitility-groovy:3.1.0') {
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user