From Noel's:

THE RAVEN SECTION
THE DEVONSHIRE HOUSE CLEAVAGE
(November, 1906)
In 1906, Mr. Russell Elliott and Mr. Alfred Mace and some other leading brethren in the Raven fellowship, were active, with some others, both Open and Exclusive, in promoting the Devonshire House Conference meetings, in London, England, which finally resulted in a few exclusive brethren accepting the principles and the communion of Open and Independent brethren. From the printed Report of the Conference, we learn that requests were received for some 700 tickets of admission, in response to the invitations, and that some of the high lights of the Conference were :
(1) The reading and distribution of a lengthy printed letter addressed to the Conference by the Independent meeting in Plainfield, N. J., IT. S. A., which had separated from the Grant Section in 1894, and signed by F. C. Jennings and two other brothers. This letter stated that: "We will own no circle of less circumference than the whole Body of Christ as the limit of our fellowship; because we will own no association of assemblies. . . . The key to the truth of God on this question has been to us this: Fellowship is not between assemblies as such, but between individual saints," etc.
(2) The Report said: "How sadly has the exercise of 'gift' been limited to our sectarian 'circles' of fellowship, and all help from 'gift' outside our particular 'circle' been refused."
Later, this Movement was absorbed by the Open Brethren, except some who agreed with R. E. and A. M., and a number of isolated Independent meetings, some of which the Open Brethren refused to absorb. At first Mr. R. Elliott was cordially welcomed amongst the Open Brethren, but after he published his pamphlet on "Household Baptism," which was extensively circulated in Canada, especially in Ontario, he sadly complained that they would no longer permit him to minister among them, and to this day some who were in the Movement say that because they teach this doctrine they are almost anathema among them. This pamphlet was replied to by Mr. C. Knapp, of Delmar, N. Y., an exclusive brother, in a tract which he published, and which is still obtainable, called "A Letter on Russell Elliott's 'Household Baptism.'" Price 5c. This Letter states that G. V. Wigram, J. G. Bellett, W. Kelly, C. H. Mackintosh, Andrew Miller, Chas. Stanley, and many other honored servants of God with brethren held and "taught believers baptism. This shows that among exclusives there is no prejudice or discrimination against anyone on account of views on baptism.

In this Conference we see "Ex-EBs" who do accept neither "circle of fellowship" of the Exclusive Brethren and the "local" assembly independecy/membership of the Open Brethren.
See: "What is the Real Fellowship of the Church of God?" by Russell Elliot ("ex-Raven"), "The Church and The Churches by W.W. Fereday" ("ex-Kelly") and "Church Fellowship" by Walter Scott ("ex-Stuart").

Noel, a Circle of Fellowship defender, points out:

No matter by what name they may be known, those who deny the circle of fellowship principle and position, are Independent brethren. It is true that even at the present time, here and there among exclusives, one may be found who denies that there is a circle of fellowship and that they are or should be "gathered on the ground of the one body," for that is the beginning of the defection, and such may sometimes remain with exclusives for months and even years before they finally act upon their denial, and join the Independents, who are like-minded. "A Divine Movement," by F. W. Grant, price 20c (Erie Bible Truth Depot, 910 French St., Erie, Pa.), is a very helpful book, which shows the scriptural authority for a circle of fellowship.

Particularly, I think this debate is very interesting, but I tend to agree with the "Ex-EBs". H.D. Woolley, one of the Coveners of this Conference, on "A Few Thoughs Resulting" brightly says:

Above all else, let us avoid anythink like assumption - calling ourselves "the remnant" etc. Personally, I shrink from using the word "assembly" in connection with a few Christians coming together in this day of brokenness, and much prefer "meeting" or "gathering". Two or three soldiers of a regiment are not the regiment. Even as to being "gathered to the Lord's name", if this is used as a term to classify certain Christians, it savours of the last four words of 1 Corinthians 1,12 - the worst kind of sectarianism. To so gather is a matter of faith at the time, not of profession, and is it not better to be in the reality of it than to make distinctive claims? Indeed the less we say about ourselves the better.

Indeed, to my mind this view is more coherent with the state of public (outward) ruin of the church, as John Nelson Darby comments on a Bible Reading: 

Ques. Is there a church now on earth or not?

J.N.D. Is there an army or not? Suppose an army not destroyed but scattered to the four winds - why, there is an army, and there is not an army; it has lost its corporate character.

It is indeed an interesting, and needed debate, even 100 years later. But is it about what “exclusives,” "Ex-EB's," “independents,” or “OBs,” say or have written, or rather - what saith the Scriptures? History is helpful to reflect on the beginnings of the past, on movements, divisions, etc - this website helps here - but where do we start when we teach the younger brethren in assemblies? Is there an assembly on earth?

Woolley’s statement that: “Personally, I shrink from using the word ‘assembly’ in connection with a few Christians coming together in this day of brokenness, and much prefer ‘meeting’ or ‘gathering,’” is strange seeing that “assemblies” (local churches) existed in NT times, all expressing the Assembly of the Living God – two or three soldiers DO represent the regiment. Paul wrote to the Corinthians, “ye are [the] body of Christ.”

I think for one, the use of the term “mystic body” has caused some confusion, because it may be used to suggest an invisible assembly - the one mystic body, including saints in heaven. But why? when the assembly (or body), as the Word shows, IS on earth. It is more the mystic unity – Christ in heaven and His body on earth that we know. J Wilson Smith wrote:

Now this, though spiritual and mystic, is surely not invisible. Each of its members, great or small, has its own place to fill in the whole, each is a “member in particular” and none can fulfil the functions, or do the work of another. All are interdependent and none can fulfil the functions, or do the work of another. All are interdependent and under the control of the Head, just as the human body acts in its varied members in perfect organic harmony and order, and it is used, in our chapter, as an illustration. This takes the truth of the “Mystery of Christ” out of the region of mere theory, and presents it in a truly practical light.

In line with his comments in the Reading André referred to above, JND wrote on the scriptural idea of the Church (I give sufficient extracts for proper context. Note: It is NOT that he is the standard, but his extensive writings do facilitate a careful checking against Scripture):

I leave aside the idea of the invisible church, an idea which is not found in the Bible. An invisible assembly is almost nonsense. There will be a universal Church manifested in glory in the day of Christ. Far from being invisible, it will be seen in all the glory of its Head. The children of God, alas! are but too often hidden in the world; and in this sense one may speak of an invisible church. In this case, where is the city set on a hill? where is the light which the Lord would not put under a bushel? To say church is to say assembly. Mr ……makes use of this word to say that, since believers have never been all assembled, there has been no assembly, nor church consequently, except by a figure which consists in speaking of a part for the whole.

I speak of the Church on earth. Now what I find in the word is an assembly on earth formed in the unity of one body by the Holy Ghost come down from heaven, the Head being in heaven.

It will be remarked, that the fact that the Holy Spirit is come down from heaven is of primary importance; because it is what establishes plainly the distinction between the state of the body viewed here below, and the heavenly state of the Church in glory. God, in the accomplishment of His counsel, will establish the Church, Christ's body, in the same glory as her Head in heaven. That is the idea of the Church in its plenitude: but, just as that has always occurred, God first set man in a position of responsibility with regard to the things which it is His will to accomplish at a later period by His power.

Commenting on Eph 3:21 – “In whom all the building fitly framed together groweth unto an holy temple in the Lord,” Darby asks, “Where? On earth it was assuredly that the tabernacle of God was formed by the presence of the Holy Ghost.”

The Church, then, is here considered objectively in all its extent possible; but it is, nevertheless, viewed as one body on earth, and as being on earth the habitation of God through His Spirit, who is come down and has united Jew and Gentile in this one body. That is the vocation in regard to which it is said, "Walk worthy of the vocation wherewith ye are called."

There is then a body, the Church, on the unity of which the apostle insists - unity which should be found on earth. The apostle speaks of the body, of the Church, of unity, as things existing and manifested on earth, not in an accidental manner, or as part of a whole which touches the earth by one of its extremities, but as flowing from the presence of the Holy Ghost sent down from heaven to form this unity on earth, to gather in one the children of God who were scattered. Though there be for those whose service is already closed the rest of the disabled, the army is no less an army; neither does it cease to be still a whole, though it be recruited from time to time with new soldiers.

Nevertheless, there is much to ponder as the public ruin of the Church, in the hands of responsible man continues, but we do have a lamp stand testimony to bear on earth until the Lord comes.