Comments for MattWeber.Org http://www.mattweber.org PYTHON, SEARCH, AND OPEN SOURCE. Sat, 05 May 2012 18:09:57 +0000 hourly 1 http://wordpress.org/?v=3.3.2 Comment on Exploring LucidWorks Enterprise with SolrCloud by matt http://www.mattweber.org/2012/05/01/exploring-lucidworks-enterprise-with-solrcloud/#comment-5956 matt Sat, 05 May 2012 18:09:57 +0000 http://www.mattweber.org/?p=194#comment-5956 Hey Mark, thanks for pointing out that it is possible to start LWE with the numShards parameter and that the LWE UI/Rest create collection commands will honor that setting. I did some testing and found that this does work, however it does not simplify any of the steps I describe in the blog post. The exception being I no longer need to specify the shard names when executing the Core Admin API commands. The numShards parameter basically only manages creating the shards in zookeeper. It does not automatically create the additional shards/replicas within LWE itself, I still need to do that manually via Core Admin API or the LWE UI/Rest API. If I start LWE with numShards=2 and don't bring up another instance of LWE and create the collection manually, I get errors while feeding. Also, to do everything using the LucidWorks Enterprise GUI/Rest api, would need to bring up 4 separate instances of LWE vs. having Shard 1/Shard 2 Replica in a single instance and Shard 2/Shard 1 Replica in another. This means I still need to drop down to using Solr's Core Admin API to set this up manually. Basically I still need to jump though all the hoops I talk about above to create a multi-shard collection with replicas. Hey Mark, thanks for pointing out that it is possible to start LWE with the numShards parameter and that the LWE UI/Rest create collection commands will honor that setting. I did some testing and found that this does work, however it does not simplify any of the steps I describe in the blog post. The exception being I no longer need to specify the shard names when executing the Core Admin API commands.

The numShards parameter basically only manages creating the shards in zookeeper. It does not automatically create the additional shards/replicas within LWE itself, I still need to do that manually via Core Admin API or the LWE UI/Rest API. If I start LWE with numShards=2 and don’t bring up another instance of LWE and create the collection manually, I get errors while feeding.

Also, to do everything using the LucidWorks Enterprise GUI/Rest api, would need to bring up 4 separate instances of LWE vs. having Shard 1/Shard 2 Replica in a single instance and Shard 2/Shard 1 Replica in another. This means I still need to drop down to using Solr’s Core Admin API to set this up manually. Basically I still need to jump though all the hoops I talk about above to create a multi-shard collection with replicas.

]]> Comment on Exploring LucidWorks Enterprise with SolrCloud by Mark http://www.mattweber.org/2012/05/01/exploring-lucidworks-enterprise-with-solrcloud/#comment-5909 Mark Thu, 03 May 2012 23:32:08 +0000 http://www.mattweber.org/?p=194#comment-5909 “When creating a new collection (with either the Admin UI or the API), you cannot yet specify the number of shards to break it up into. If this is something you need to do, please contact Lucid Imagination for assistance.” Yeah, I checked on this and this was pointed out to me. It's a lapse in the doc - something else was meant. It will be corrected. The default UI and API calls for collection management will work. I can ask that we make sure to explicitly document that collection management still works when using SolrCloud functionality. "What I want to do is create a single collection that is composed on two partitions (shards) and a single replica." If you want 2 shards, simply use a numShards of 2. Once you start the first instance, it will be part of shard1. Start another and it will be part of shard2. Start another and it will be a shard1 replica. Start a 4th instance and it will be a shard2 replica. You just give the number of "partitions" and then instances evenly become replicas for each shard. We plan on adding further ways to control this, but that is the initial knob. “When creating a new collection (with either the Admin UI or the API), you cannot yet specify the number of shards to break it up into. If this is something you need to do, please contact Lucid Imagination for assistance.”

Yeah, I checked on this and this was pointed out to me. It’s a lapse in the doc – something else was meant. It will be corrected. The default UI and API calls for collection management will work.

I can ask that we make sure to explicitly document that collection management still works when using SolrCloud functionality.

“What I want to do is create a single collection that is composed on two partitions (shards) and a single replica.”

If you want 2 shards, simply use a numShards of 2. Once you start the first instance, it will be part of shard1. Start another and it will be part of shard2. Start another and it will be a shard1 replica. Start a 4th instance and it will be a shard2 replica.

You just give the number of “partitions” and then instances evenly become replicas for each shard.

We plan on adding further ways to control this, but that is the initial knob.

]]> Comment on Exploring LucidWorks Enterprise with SolrCloud by matt http://www.mattweber.org/2012/05/01/exploring-lucidworks-enterprise-with-solrcloud/#comment-5841 matt Tue, 01 May 2012 14:54:46 +0000 http://www.mattweber.org/?p=194#comment-5841 That would be great but I was unable to find any documentation supporting this, thus the reason I needed to drop down to Solr Core Admin API. "When creating a new collection (with either the Admin UI or the API), you cannot yet specify the number of shards to break it up into. If this is something you need to do, please contact Lucid Imagination for assistance." http://lucidworks.lucidimagination.com/display/lweug/Using+SolrCloud+in+LucidWorks http://lucidworks.lucidimagination.com/display/lweug/Collections Also, you say numShards is for replicas. What I want to do is create a single collection that is composed on two partitions (shards) and a single replica. Is there a system property that specifies the number of partitions I want? The definitions of terms used in SolrCloud is confusing. That would be great but I was unable to find any documentation supporting this, thus the reason I needed to drop down to Solr Core Admin API.

“When creating a new collection (with either the Admin UI or the API), you cannot yet specify the number of shards to break it up into. If this is something you need to do, please contact Lucid Imagination for assistance.”

http://lucidworks.lucidimagination.com/display/lweug/Using+SolrCloud+in+LucidWorks
http://lucidworks.lucidimagination.com/display/lweug/Collections

Also, you say numShards is for replicas. What I want to do is create a single collection that is composed on two partitions (shards) and a single replica. Is there a system property that specifies the number of partitions I want? The definitions of terms used in SolrCloud is confusing.

]]> Comment on Exploring LucidWorks Enterprise with SolrCloud by Mark http://www.mattweber.org/2012/05/01/exploring-lucidworks-enterprise-with-solrcloud/#comment-5840 Mark Tue, 01 May 2012 14:45:13 +0000 http://www.mattweber.org/?p=194#comment-5840 Hey Matt - a couple things I'd point out - first, the numShards param is the key to controlling the number of replicas. Second, with LWE, you do not need to drop down to Solr and make all these calls at all. I'm trying to find where in the doc it says you can not use the admin or rest api for this, but I don't see it. In fact you can! Simply use the LWE collections API to create a new collection, or create a new collection in the admin UI. One of the params you can pass if using the ReST API is numShards. That is not available from the UI yet (it uses whatever numShards was passed as a sys prop on startup. The integration for 2.1 is certainly early - but you don't have to jump through all these hoops at all. Hey Matt – a couple things I’d point out – first, the numShards param is the key to controlling the number of replicas. Second, with LWE, you do not need to drop down to Solr and make all these calls at all. I’m trying to find where in the doc it says you can not use the admin or rest api for this, but I don’t see it. In fact you can! Simply use the LWE collections API to create a new collection, or create a new collection in the admin UI. One of the params you can pass if using the ReST API is numShards. That is not available from the UI yet (it uses whatever numShards was passed as a sys prop on startup.

The integration for 2.1 is certainly early – but you don’t have to jump through all these hoops at all.

]]> Comment on Solr for WordPress 0.2.0 Released by Peter http://www.mattweber.org/2009/05/13/solr-for-wordpress-020-released/#comment-5356 Peter Mon, 16 Apr 2012 04:39:53 +0000 http://www.mattweber.org/?p=172#comment-5356 Great plugin! I have a simple question. I want to retrieve all results because i want to start with a complete facet list. Normally on Solr i would use a query (*:*) with a limit of 0. Thsi doesn;t return the results but gives me a complete facetlisting. But this didn't work in thye plugin. How can i get all results/facets ? Great plugin!
I have a simple question. I want to retrieve all results because i want to start with a complete facet list. Normally on Solr i would use a query (*:*) with a limit of 0. Thsi doesn;t return the results but gives me a complete facetlisting. But this didn’t work in thye plugin.
How can i get all results/facets ?

]]> Comment on ElasticSearch Mock Solr Plugin by peter fox http://www.mattweber.org/2011/12/14/elasticsearch-mock-solr-plugin/#comment-4571 peter fox Fri, 23 Mar 2012 03:45:36 +0000 http://www.mattweber.org/?p=190#comment-4571 Hi Matt, We have a dev site running on wordpress and I'd like to use solr and your plugin to seed a spider off links or content. can you help? Hi Matt,
We have a dev site running on wordpress and I’d like to use solr and your plugin to seed a spider off links or content. can you help?

]]> Comment on Solr For WordPress on GitHub by alex http://www.mattweber.org/2011/03/29/solr-for-wordpress-on-github/#comment-3152 alex Mon, 23 Jan 2012 16:30:34 +0000 http://www.mattweber.org/?p=189#comment-3152 Has anyone had any success with faceting by custom taxonomy in the search results page? I see that my custom taxonomies are indexed but the facet options appear to only be Category, Author, and Tag. Has anyone had any success with faceting by custom taxonomy in the search results page? I see that my custom taxonomies are indexed but the facet options appear to only be Category, Author, and Tag.

]]> Comment on Spotlight CHM Metadata Importer by Jens http://www.mattweber.org/2007/03/11/spotlight-chm-metadata-importer/#comment-3116 Jens Sun, 22 Jan 2012 17:31:37 +0000 http://www.mattweber.org/2007/03/11/spotlight-chm-metadata-importer/#comment-3116 CHMporter is PPC. Chimp's Spotlight plugin is also PPC. Any way to make Matt's plugin work in say 10.5+? CHMporter is PPC. Chimp’s Spotlight plugin is also PPC.

Any way to make Matt’s plugin work in say 10.5+?

]]> Comment on Solr For WordPress on GitHub by Dany http://www.mattweber.org/2011/03/29/solr-for-wordpress-on-github/#comment-1188 Dany Tue, 08 Nov 2011 14:24:19 +0000 http://www.mattweber.org/?p=189#comment-1188 I am also having some problems with committing the pages and posts. Solr is running and the settings are good. I am having solr v3.4.0 and wp 3.2.1 If anyone has a solution, pls let me know. I am also having some problems with committing the pages and posts. Solr is running and the settings are good.

I am having solr v3.4.0 and wp 3.2.1

If anyone has a solution, pls let me know.

]]> Comment on Solr For WordPress on GitHub by matt http://www.mattweber.org/2011/03/29/solr-for-wordpress-on-github/#comment-1165 matt Mon, 07 Nov 2011 03:31:54 +0000 http://www.mattweber.org/?p=189#comment-1165 Have you tried the version on GitHub, there has been work on multisite support recently. Have you tried the version on GitHub, there has been work on multisite support recently.

]]>